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FOREWORD 



The FAA has been increasingly concerned about the National Train- 
ing Programs for development of En Route and Terminal controllers. 
Divergent views have been expressed concerning location and duration 
of this training, the order and content of the curriculum, the use and 
capabilities of simulators and the degree of centralization that is 
desirable. This report addresses these questions. Where answers are 
not available now, a way of obtaining the answers is offered. Cost is 
the dominant measure used in comparing alternatives. 
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and the preparation of this report, James J. Bagnall performed an 
early investigation of the simulation capabilities of the existing 
ATC systems and the associated improvement program. Vernon I. Weihe, 
consultant, of Arlington, Va. , reviewed the capabilities of digital 
computer simulators , particularly as they could be used for the 
ARTS II system. The data obtained in the facility survey that was 
performed were processed and developed by Elizabeth Ratigan of the 
IDA Computer Group. Beth A. McClain helped to prepare the survey 
data and also helped in using the results to develop program costs. 
Their efforts are gratefully acknowledged, 

A draft of this report was reviewed by Joseph G. Colmen, presi- 
dent of Education and Public Affairs, Inc., and by Stuart H, Starr and 
John J, Metzku, both of IDA. Their helpful suggestions and revisions 
have made an important contribution to this report. 

Finally, the assistance and support of Arnold Corradino, James I. 
Moore, and H. Douglas French, Office of Personnel and Training, FAA, 
are greatly appreciated. Gratitude is due for their efforts to obtain 
much of the' necessary information for this study. 
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SUMMARY 



A • OBJECTIVE 

The objective of this study is to provide information useful for 
improving the air traffic controller training system of the U.S. Fed- 
eral Aviation Administration (FAA) . Special consideration is given 
to the cost and duration of training and to the ability of the train- 
ing system to handle various loads. Key issues are the benefits of 
centralized and noncentralized training and the need for simulators* 

B. FINDINGS 

The principal findings of the study are summarized here, grouped 
according to the tasks assigned by the agreement for the study. Evi- 
dence in support of each finding appears in the report on the pages 
cited in parentheses. 

Task 1. Training Load 

1,1. The hiring requirement for new^ air traffic controllers is 
highly dependent on the separation rate of controllers and 
the attrition rate of trainees. The present attrition rate 
of trainees (FY 1974) is ^3 percent for the en route option 
and 38 percent for the terminal option. The average annual 
separation rate for the last 13 years has been 5.1 percent 
for en route controllers and 3,3 percent for terminal con- 
trollers. Should these latter separation rates prevail 
into the future, the^ new hires needed to fill FAA projec- 
tions of required controller positions are: 
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Trainee FY 1975 ^ ^l^'^ . , 

Attrition Rate En Itoute Terminal En Itoute Terminal 

Present Rate 1110 1350 1160 1480 

1/2 Present Rate 810 1040 840 1130 

1/4 Present Rate 710 920 . 740 1000 

(pp. 11-21) 



Task 2. Training Process 

2.1. Improved selection criteria are available that could reduce 
the number of controllers who fail during training. If 
failures could be eliminated completely, the cost of training 
would be reduced 33 percent for the en route option and 22 
percent for the terminal o tion. (pp. 25-26) 

2.2. Objective measures of performance of air traffic control- 
lers have been developed by the FAR but are not in use at 
present. These measures are important for determining the 
competence of controllers during and after training, (pp. 
27-30) 

2.3. The presents curriculum provides the information needed by 
controllers to do their work. However, questions about the 
amount of time given to various segments of training, the 
order of training (e.g., non-radar training followed by 
radar training) , and the value of training on prototype 
sectors can be resolved best by an experimental approach 
which would use the foregoing objective performance measures, 
(pp. 26-27) 

2.4. There is substantial evidence of lack of standardization 
throughout controller training. An FAA office should be 
vested with a positive role of certification of both field 
facility training programs and the qualification of individual 
trainees, (pp. 26-27, 61) 

Task 3. Simulation 

The simulators in the National Airspace System, Stage A 
(NAS-A) and the Automated Radar Terminal System (ARTS) III 
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appear sufficiently realistic and adequate for radar train- 
ing of controllers, although, they have some limitations, 
e.g., analog information cannot be simulated on the ARTS 
III system. These limitations could be remedied by develop- 
ment of inexpensive software and hardware, some of which is 
already under way. The simulators at the facilities are 
usable for proficiency, remedial, and supplemental training. 
Because performance is not measured objectively at present, 
a program of software development to accomplish this should 
be introduced. A simulation capability for proficiency 
testing and supplemental training should be introduced in 
• the ARTS II. (PP- 35-41) 

3.2. The cost of air traffic control simulators is a small part 
of the training cost for developmentals (less than 5 per- 
cent) . A full simulation capability should be introduced 
at a centralized training facility, presumably the FAA 
Academy, (pp. 53-55) 

3.3. The training requirements introduced by improvements in 
future systems should be identified early in the develop- 
ment cycle. This is important in order to procure hardware 
and software for simulation and to develop proper training 
programs in a more timely fashion, (pp. 57-58) 

Task 4. Training Method 

4.1. Within the accuracy of the study's cost estimation and 

training program specification, there is little difference 
in the costs of actual trainir of developmentals on a 
centralized or nori centralized basis for either controller 
option. 

The duration of the developmental period has a strong 
impact on the cost of training, and shortening of this 
period can save as much as $48,000 for training each con- 
troller. With full-time academic training, screening can 
be accomplished sooner, and developmentals' unproductive 
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time can be reduced. A centralized facility would have 
greater capability for full-time training than an operating 
control facility/ (pp. 43-51) 

General Findings 

5.1. The total controller training budget is substantially 
greater than is explicitly identified at present. The 
actual amount should be determined and the application to 
purposes of training should be reviewed regularly, (pp. 
55-5G) 

5.2. New R&D on controller selection, performance measures, 
order of training, and training techniques is warranted. 

A focal point should be established to coordinate all per- 
sonnel-related R&D in FAA, including ongoing research, and 
to respond to findings. (pp\ 58-59) 

5.3. Fluctuations in hiring can and should be reduced. More 
efficient use of the Academy would result. At current 
staffing and training leads, approxinately $1 million could 
,be saved annually by smoothing trainee enrollments. 

(pp. 59-52) 
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I. INTRODUCTION 



A. OBJECTIVE AND TASKS 

The objective of this study is to provide information useful for 
improving the air traffic controller training system of the U.S. 
Federal Aviation Administration (FAA). Special consideration is 
given to the costs and duration of training and to the ability of 
the training system to handle various loads. Key issues are the 
benefits of centralized and noneentralized training. The study ^ 
addresses the following tasks (paraphrased from the work statement 
reproduced in Appendix A): 

• Review the evolution of the existing national programs 

for the qualification, refresher, proficiency maintenance, 
and supplementary training of en route and terminal control, 
speciali sts. 

1. Develop estimates of the number of individuals who will 
require various types of qualification and supplementary 
training. 

2. Access the adequacy of current specifications for training. 
Compare (a) the content and relevance of present training 
courses and (b) the methoJs used to establish competence at 
course completion and the current specifications for train- 
ing. Examine the extent to which the current specifications 
for training are based upon analyses of on-the-job perform- 
ance and the degree to which qualification standards have 
been validated against operational performance data. In- 
cluded would be an analysis of the need for, and a descrip- 
tion of, the requirements for standardization and quality ^ 
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control in terms of tho implication or impact, or the lack 
thereof, on training. 

3, Appraise the capabilities and limitations of the state 
of the art of simulation devices relevant to air traffic 
control training. Describe and analyze the applicable 
simu]ation, on-line, and classroom facilities that may 
be required for training the anticipated loads. Include 
the influence of such factors as number, type, and com- 
plexity of equipment, location of the training facil- 
ities, and sequence of progression through training 
blocks upon the number of individuals who may bo trained 
per unit time, the duration of training, and the total 
annual cost of the training program. The purpose of the 
analysis is to provide information needed to make deci- 
sions concerning the number, type, and location of 
training facilities and equipment. 

4. Identify and evaluate the alternative methods currently j 
and potentially available in which training needs could 

be satisfied. 

B. METHODOLOGY AND MANNER OF PRESENTATION 

a* 

Standard fcsearch procedures were us>ed for much of the study; e.g., 
training materials were, examined, visits were made to FAA facilities, 
interviews and discussions were held, regional and headquarters 
reports were reviewed, research reports were studied, statistical 
information was collected. Later, a survey was undertaken of all 
en route and terminal facilities because specific data on training 
at these facilities were needed. A seven-page questionnaire was 
constructed and disseminated. All the centers and over 80 per- 
cent of the terminal facilities responded. These responses became 
an important source of data for this study and may have value to 
other FAA activities. 
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The main body of this report consists of chapters arranged in 
the order of the tasks listed above, followed by a chapter that 
covers a number of important aspects developed in the process of 
performing the study. Nine appendices contain much of the support- 
ing data and information. 

C. THE AIR TRAFFIC CONTROL SYSTEM 

The purpose of air traftic control (ATC) is to ensure safe 
and efficient movement of aircraft. Control of aircraft in the 
National Airspace System (NAS) is done from the gi'ound, and it is 
designed to keep aircraft separated from each other and to expedite 
the flow of traffic. Control facilities develop information from a 
variety of sources, including long-range surveillance radars, local 
airport radars, adjacent control areas, and by direct vision from 
towers. Air traffic control is exercised at terminals and between 
terminals. Control between terminals is called cn route control. En 
route control in the continental United States is distributed among 2 
air route traffic control centers (ARTCCs) for aircraft operating on 
instrument flight rules (IFR). Terminal control facilities can be 
divided into those capable of handling traffic operatiiig on IFR and 
those that can handle only aircraft operating under visual flight 
rules (VFR). Controllers are. usually classified by the kind of facil 
ity in which they operate, i.e., en route, IFR terminal, and VFR 
terminal. Training programs for controllers are designed for each of 
these specialties. 
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II • EVOLUTION OF THE NATIONAL TRAINING PROGRAM 



This chapter reviews the evolution of the existing national 
program for the qualification, refresher, proficiency maintenance, 
and supplementary training of en route and terminal control special- 
ists. 

The objective of the national training program is to equip and 
retain controllers who can make the ultimate decisions necessary to 
maintain the safe, expeditious, and orderly flow of air traffic. At 
times the controllers who operate this man-machine system perform 
under great stress; often they operate under great boredom. The 
character of this workload provides the basis for selecting and 
training air controllers. 

ATC services were initiated in November 1941. The training 
program started in the same year and, in the next two years, seven 
training centers were established in key locations throughout the 
country. Wartime flying demands caused the service to be considerab] 
expanded. Women were also recruited, and by the end of World War II 
they comprised one-third of the controller work force. In contrast, 
only 2.2 percent of enrollments for ATC training in 1969 were women 
(Cobb et al., 1972), The pay of women has always been the same as^ 
that of men in the Air Traffic Service (ATS). 

After World War II, the regional training schools were closed, 
and many instructors moved to the Aeronautical Center established 
by the Civil Aeronautics Administration (CAA) in 1945 at Oklahoma 
City. However? most training reverted to an on-the-job training r 
(OJT) method. Centralized training was authorized in 1956 by the 
CAA at what became in 1959 the FAA Academy. When the ATC work { 
force expanded in 1959-63, this centralized training comprised >• 

9 



19 



some 4 to 8 weeks of training on the so-called basic airman sub- 
jects (e.g., flight navigation, communication, maps and flight 
plans). At that time the Academy program performed a primary screen- 
. ing function by determining the aptitude of prospective air traffic 
controllcx»s while at the school. 

By 1963, enrollment at the Academy had doclinod to 76 percent 
of capacity. Although plans wore developed to increase the depth 
and amount of centralized training, ATC training at the Academy was 
discontinued in 1963 when recruitment of new controller candidates 
fell to a low level. The primary function of the Academy then bo- 
came the preparation and distribution of training materials to the , 
facilities which conducted the qualification, refresher, and sup- 
• lementary training programs. 

Centralized training of the G- to 8-wcek variety was reinsti- 
tuted in 1968, primarily for screening purposes. In 1970, a major 
part of the Phase II portion of the en route training program was 
inaugurated at the Academy, and radar control training was accom- 
plished by using the simulation facilities at NAFEC. This latter 
expedient was discontinued in 1972. Comparable portions of term- 
inal control specialist training were performed at the Academy 
beginning in 1971. However, lack of adequate simulation facilities 
prevented effective radar control training, and it still does. The 
present Terminal Training Program was adopted in 1972. 

Refresher, proficiency maintenance, and supplementary training 
have always been under the control and operation of each facility. 
The standards are established at the Academy, which also prepares 
self -study materials. For example, supplementary training associated 
with the introduction of automation has generally been performed 
on an expediency basis at each facility. 
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III. PROJECTED TRAINING LOAD 



This Chapter develops estimates of the number of individuals 
who will require various types of qualification and supplementary 
training. Estimates arc provided for three classes of trainings- 
en route, terminal, and supplemental. A variety of data sources 
have been used, including the recent IDA survey of training at 
centers and terminals described in Appendix B. The estimates arc 
based on the relationship between controller positions, attrition, 
and training losses and reflect controller productivity improvements 
Some problems inherent in translating the necessary hiring to an 
efficient training program arc presented. 

A. METHOD 

The method used to develop the estimates of the developmental 
training load was as follows: 

a. Determine from FAA (1973b) and FAA (1974f ) the linear trend 
of traffic handled by centers and terminals. 

b. Take the linear trend through the actual number of control- 
ler positions in the last 6 years (FAA RIS MIN 3300-5 
reports) and the estimates of requirements through 1985 
(Office of Aviation Economics, FAA, 1974c). The projected 
productivity improvements can be observed. 

c. Examine the history of the separation rates of controllers 
(FAA, 1971b and RIS MN 3300-5 reports) to estimate a future 
separation rate. 

d. Estimate the attrition of air traffic controllers from 
the IDA survey for FY 1974. [Similar attrition rates 
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were reported in the Corson Report (Corson et al., 1970) 
and more recently in the Great Lakes Region (Hollinger, 
1974).] 

e. Use an elementary difference equation to predict necessary 
hires, and hence the future developmental training load. 

Another method of estimation was initially tried. In that 
method, an attempt was made to construct an estimate from the sum ^ 
of the requirements at each individual facility and thereby reflect 
their present situation and trends. However, there was such vari- 
ability in the facility data (Appendix B) that consistent projections 
were infeasible. This approach is unworkable without detailed know- 
ledge of the separation, hiring, and training situation prevailing 
at each facility. 

B. EN ROUTE 

The history and latest projection of IFR traffic handled (over- 
flights plus departures multiplied by two) for the United States is 
shown in Fig. 1. The straight line is a least -squares fit to these 
data. Also shown are the numbers of all en route controller posi- 
tions for past years and the provisional estimates for future years. 
These data apply to the end of the fiscal year. A straight line is 
again fitted to the controller data for smoothing purposes. The 
data smoothed by the linear fits are recorded in Table 1. 

The straight-line fit avoids the discontinuities in the esti- 
mates of controller numbers developing from different sources at 
different times and from changes in productivity estimates. 

Experience concerning ARTCC separations for the en route con- 
trollers is shown in Fig. 2. Considerable fluctuation occurs from 
year to year (from 2.1 percent in FY 1965 to 11.2 percent in FY 
1970). The average separation rate is 5.1 percent. 
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TABLE 1. SMOOTHED TRAFFIC AND EN ROUTE CONTROLLER ESTIMATES^ 



Year 

1970 
1972 
1974 
1976 
1970 
1980 
1982 



IFR 
Traffic 
(millions ) 

19.2 
21.5 
23.8 
26.0 
28.3 
30.5 
32.8 



All 
En Route 
Controllers 
(thousands ) 

10.0 
10.2 
10.4 
10.6 
10.8 
11.0 
11.2 



Gross 
Productivity 
(IFR/Controller) 

1920 
2108 
2288 
2453 
2620 
2773 
2929 



Annual 
Productivity 
Increase 
(percent) 



4.0% 

4.2 

3.G 



3.4 
2.9 
2.8 



Data Source: Least-squares fit in Fig. 1. 
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13 YEAR A^/KRAGE 5.1% 
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IO-IS-74-15 



1965 1970 
FISCAL YEAR 



1975 

Source: FAA AMN-22 



FIGURE 2. ARTCC Controller SeparaHon Rate 
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The loss rate of developmenuls during the training period is 
substantially larger tten the later loss rate of journej^en. The . 
Corson C«nittee (Corson ot al., 1970) reported attrition as a per- 
centage of hires to be: 

Year Attrition 
- 1967 17.4% • 

1968 ^^'^ 

1969 ^^'^ 

The IDA survey found an overall attrition (all phases) to bo some- 
what lower-14.3 percent for en route in FY 1974. This corresponds 
to a loss rate of approximately 43 percent over the entire deyelop- 
' mental period (Appendix G, Table G-13). 

A projection to the future training load can be made by us- 
ing a difference equation relating hires with numbers of control- 
lers required and separation and training losses, 

Hy = C(l-. r)Cy.,.l-Cy]/(l-s), (1) 

where H represents the number of new hires in year y 

r is the separation rate of journeyman controllers 
s is the loss rate of devolopmentals during training 
C is the number of controllers required in year y. 
^ This reflects changes in controller productivity in 
year y. The number of controllers required is shown 
in Table 1. 

The required hiring to meet retirement, training losses, and 

traffic .controller position increases is derived for en route 

centers by using Eq. 1 and is shown in Pig. 3. The smoothe con- 

^^4-o r>r^o ii<;pd To meet the increasing tratric 
troller requirements are usea. >"^^^ 

demands. 103 additional positions per year -^7^; J° ■ 
training losses alone, 181 employees must be hired for these addi 
ional !o3 positions. The refining hires are to replace separation 
losses. Also shown in Fig. 3 are the effects of reduced attrition 
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during training, from the FY 1974 rate of 43 percent to assumed val- 
ues of 20 percent, 10 percent, and zero (with separation rate kept 
constant at the present 5 percent). The training load obviously is - 
much more sensitive to separation rate. In Fig. 4, the necessary ' 
hires to meet 7.5 percent, 5.0 percent, 2.5 percent, and zero separa- 
tion rates are shown (in this case, with developmental loss rate kept 
constant at its present 43 percent.) Also shown, as individual points 
are the actual en route controller hires for the last 5 fiscal year^. 
Although the latter vary greatly, the prediction is withir. Lm^ 
torical range. 

The great sensitivity uf the training load ro separation rates 
and developmental loss rates is important. A training load of less 
than 1100 new developmentals each year would seem feasible for the 
en route option as long as the separation rate of journeyman con- 
trollerb does not exceed 5 percent. Productivity changes have a 
small effect on the number of new hires required. 

C. TERMINAL 

In the same manner, projections have been made for controllers 
in the terminal option. Figure b shows the history and latest pro- 
jection of tqtal terminal operations in the United States. Again, 
the straight line is a least-squares fit to these data. Also shown ^ 
are the history and projections of IFR operations at terminals. It 
is evident that most oE the growth of terminal operations will be 
of the IFR type, VFR increases being considerably less. Finally, 
the history of controller positions is shown along with the recent 
FAA estimates. The data, smoothed by the linear fits, are recorded 
in Table 2. The changes in controller productivity are also shown. 

The experienced separation rate of terminal controllers is 
shown in Fig. 6. Fluctuation has been less than for en route con- 
trollers, and the average separation rate of 3.3 percent is also 
Itrss . 

The loss rate of developmentals in the IFR terminal option in 
FY 1974 was 38 percent. 
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FIGURES. Terminal Traffic Load and Controller Positions 
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TABuE 2. SMOOTHED TRAFFIC AND TERMINAL 
CONTROLLER ESTIMATES 



All Gross Annual 

Total IFR Terminal Productivity Productivity 

Operations Operations Controllers (all operations/ Increase 

Year ( mi H ions) (millions) (thousands) controller) (percent) 



1970 


51.5 


18.0 


8,578 


6,004 




1972 


57.7 


21.5 


9,535 


6,051 


0.39% 


1974 


63.8 


25.1 


10,491 


6,081 


0.25 


1976 


70.0 


28.7 


11,447 


6,115 


0.25 


1978 


76.1 


"32.2 


12,404 


6,135 


0.25 


1980 


82.3 


35.8 


13,360 


6,160 


0.20 


1982 


85.4 


39.3 


14.317 


6,172 


0.20 



a 



Data Source: Fig. 5. 
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From Eq. 1, the projections for new hiring to meet separations, 
training losses, and expanded operations, with allowance for expected 
productivity improvements, are shown in Figure 7. The projections 
have been smoothed. To meet traffic expansion alone, approximately 
480 additional terminal positions are required each year. When FY 
1974 developmental loss rates (30 percent) are introduced, this re- 
quirement rises to 771. The remaining hires are to replace controller 
separations. 

Figure 8 shows the significance of separations for hiring re- 
quirements. The training load projection is very sensitive to the 
separation rate. 

As has already been observed, growth uf controller complement at 
VFR towers is expected to be slower. As approximately 20 percent of 
controllers operate at VFR facility, it can be expected that approx- 
imately 250 VFR controllers will be needed in FY 1975 and about 300 
l! in FY 1980. 

With present separation and developmental loss rates, a terminal 
option training load of 1300 in FY 1975 and 1500 in FY 1980 can be 
anticipated. As will De seen in later sections, these estimates can 
be significantly modified. 

D. SUPPLEMENTAL 

As specified, and as confirmed by the IDA survey, nearly all con- 
trollers uncJergo supplemental proficiency training during a year. In 
FY 1974, 94 percent of en route controllers and 92 percent of IFR 
terminal controllers undertook such training. The average duration 
of such training as performed at each facility was 42 hours for 
centers and 53 hours for terminals, averaging over a week for each 
controller. Refresher training involved 90 percent of en route con- 
trollers and essentially all terminal controllers. 

Some form of remedial training in FY 1974 was required for 
approximately 4 percent of en route controllers and 10 percent of IFR 
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ttfcmindl controllers, averaging 22 and 14 hours, respectively (Appen- 
dix G, Tables G-13, G-14, and G-15). 

In sum, the equivalent of one to two weeks of proficiency train- 
ing is given to the average journeyman controller each year. 

E . FINDINGS 

The hiring rt-Muirement for new air traffic controllers is very 
sensitive to the separation of controllers and the attrition rate of 
trainees. The average annual separation rate for the last 13 years 
has been 1>.1 percent for en route controllers and 3.3 percent for 
lermina.1 controllers. Should those separation rates prevail into 
tht-' luture, the new hires needed are as follows: 

FY 1975 FY 1980 

En Route Terminal En Route Terminal 

Present Student Attrition 1110 1350 1160 1480 

One-Half Present Attrition 810 1040 840 1130 

One-Quarter Present Attrition 710 920 740 1000 

The pro:,.-nL student attrition (FY 1974) is 43 percent for en route 

contro-l lers ami 38 percent for terminal controllers. 
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IV. PRESENT CONTROLLER TRAINING PROGRAMS 



This Chapter examines the adequacy of current specifications 
for training, the content and rcievancc of present training courses, 
and the methods used to establish competence at their completion. At 
the outset, selection procedures arc reviewed because of their impact 
upon the likelihood that a new hire can successfully complete his 
training, as well as upon the quality of his performance on the job. 
Then, attention is given to the relevance of present training courses 
to the work actually performed by air traffic controllers. 

This leads to an investigation of the meLhods used to measure 
performance and to establish competence on the job. The role of 
standardization and quality control is also addressed. 

A . SELECTION 

The FAA has long recognized the Importance of selecting qualified 
controller candidates and has conducted many studies to improve its 
selection criteria. The present selection procedure includes a Civil 
Service Commission aptitude test, evaluation of amount and type of 
experience, an age limitation of 30 years, reference and background 
checks, medical examinations (both physical and psychological), and 
an interview. A number of studios have examined the relation of 
test scires at time of selection to success in training and to length 
of service. Unfortunately, little effort has been made to relate 
selection criteria to objective information about the performance of 
journeyman controllers. 

FAA studies (by Education and Public Affairs, Inc., and the 
Civil Aeromedical Institute, for example) indicate, however, that 
selection procedures can be improved by incorporating tests which 
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measure behaviors required of controllers on the job. Experiments show 
that tests which measure skills in psychomotor coordination and ability 
to perform several tasks simultaneously would select superior journey- 
men, as judged by supervisors (Education and Public Affairs, 1972, 
and Chiles et al., 1972). Further improvements are possible by assign- 
ing newly hired personnel to the en route, IFR, or VFR option on the 
basis of their test scores. Such improved selection procedures would 
be expected to reduce the number of candidate controllers who fail to 
complete their training. (See Appendix D.) 

Improved selection methods" have monetary implications. Cost 
savings can accrue from improved selection and assignment (Appendix G, 
p.. 242). If the attrition losses during training could be completely 
eliminated, the cost of en route controller training would be reduced 
by 33 percent and the cost of terminal controller training would be 
reduced by 22 percent. If the trainee losses could be reduced by 
50 percent, training cost reductions of approximately 16 percent for 
en route training and 11 percent for terminal training would result. 



B. RELEVANCE (see Appendix F) 

The curricula of both the National En Route and the Terminal Air 
Traffic Training Programs specify in great detail the information 
and procedures to be taught during training and the methods of assur- 
ing their accomplishment at each phase. These programs have frequent 
and vigorous reviews. At the same time, there is evidence (e.g.j p-. 
184) that training is not standardized at the facilities, that train- 
ing times given to various topics differ between facilities, that some 
procedures specified by the curriculum are inconsistent with others in 
the curriculum,. that qualification standards are interpreted differently 
at various facilities, and that some teaching materials do not conform 
to current operational practices. Such a situation will continue as 
long as there does not exist a fully supported mechanism to ensure 
adherence to program standards or to ensure uniform changes in the 
program across the system. 
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The relevance of training to operational requirements is main- 
tained in ooveral ways. Only experienced controllers constitute, the 
training corps. Evaluation surveys of courses are conducted by ..the 
Air Traflit Branch (AAC-930), Corrections are generally implemented 
v/hun within program budget. Improvements to the present curriculum 
and training "|)rocedures are possible, as discussed in the next sec- 
tion. ThQ^Q can be tested and appraised in an objective fashion by 
means of valid measures of performance also discussed in the next 
section. Questions about present training include: 

!• Is the time allocation appropriate for each training segment? 

2. Is the order of training appropriate? In particular, if 
radar is less complex to learn than non -radar, should 
radar training follow non-radar training, as at present? 
Learning theory strongly suggests that one learns more 
effectively when proceeding from simple to complex tasks, 
rather than in the reverse order. 

3. V^hat is the value of training on prototype sectors (e.g.. 
Tango) rather than on actual ones? 

Th(. FAA training program can be made more efficient and effective 
by adopting an experimental method to evaluate alternative concepts 
about what has to be taught and how to do so. The associated measure- 
ment capability needed for such evaluations is now close at hand with- 
in the FAA (Appendix E), This could substantially improve the com- 
mendable effurt, now primarily subjective, that is accorded the reso- 
lution of major issues about how best to train new controllers and 
how to assure competence among journeymen, 

L. MEASURING THE EFFECTIVENESS OF TRAINING AND PERFORMANCE (see 
Appendix E) 

Objective performance measures have important implications for 
the evaluation of alternative methods of training. They also have 
important applications, as mentioned above, in the selection of air 
traffic controllers, in the evaluation of controllers^ performance 
on the job, in determining controllers^ maximum useful workload for 
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purposes of sector design, and for testing controllers' performance on 
proposed improvements to the air traffic system. Despite such value, 
objective performance measures are not in use at present, although their 
feasibility has been demonstrated in work at the National Aviation 
Facilities Experimental Center (NAFEC) (Buckley et al., 1969, 1972). 

As journeymen, controllers are assessed for proficiency by 
periodic written examinations, over-the-shoulder evaluations, and 
the Employee Appraisal Records (EARs) required by the Civil Service 
Commission. The purpose is to determine whether proficiency is 
acceptable; the criterion is pass or fail. Failure would indicate 
the need for refresher, remedial, or supplemental training, or, in 
the extreme case, elimination from the service for unacceptable per- 
fortnance. I'his method of measuring performance, however, does not 
establish the level or degree of proficiency. All controllers who 
pass the tests are not equivalent in performance. Such measures make 
it difficult, if not impossible, to establish whether one method of 
training is more effective than another, quite apart from consider- 
ations of accuracy, reliability, or objectivity of measurement. There 
are no scores (except, perhaps, in written tests) which can measure 
actual performance on the job. Among the results of inadequate 
, measures of perfomance are possible inefficiencies if developmentals 
are trained longer than really required. Training is oriented to 
provide the required number who can just qualify. 

There are several reasons to believe that there are significant 
limitations to the present use of ratings in general and to the over- 
the-shoulder rating in particular. These relate to the subjective- 
nature of the. rating procedure and problems of inter-rater agreement. 
A rating describes one person's judgment of another, and it can be 
influenced by a variety of causes, such as the extent of the super- 
visor's knowledge of the employee's performance, differing standards 
between c.upervisors about what constitutes adequate performance, and, 
in a subtle way, social and personality factors (Appendix E). In 
addition, the lack of standardization in the traffic samples used to 



28 



ERIC 



38 



evaluate controllers^ performance makes comparisons between persons, or 
comparisons of the same person over time, of questionable utility. 
Variability in conditions of observation, including such factors as 
the density and complexity of traffic, and in the duratiox* of the 
observation period have also been shown to degrade over-the -shoulder 
ratings (Buckley et al., 1969)* 

Recent studies for the FAA by the System Development Co-nporation 
(SDC, 1971, 1972a-d, 1974a-f) have developed new controller rating 
procedures that are more carefully specified and more relevant to the 
important aspects of the controller's job than are those used" up to now. 
To the extent that controllers are evaluated on behavior that is directly 
observable, explicitly described, and truly related to job performance, 
improved reproducibility of subjective measures can be expected, 
although this has not yet been demonstrated. 

It is feasible td rn^easure a controller's performance in an ob- 
jective way; that is, in a way that praiuces quantitative scores, 
uncontaminated by human error, on various characteristics of per- 
formance. Such major characteristics as speed and accuracy of response, 
maximum number of aircraft handled, maintenance of .separation stand- 
ards (measured in time and distance), and the- like can be measured. 
The prototype for doing this is a recording and scoring module 
attached to a dynamic simulation of the air traffic controller's 
console and work environment. It could also be attached to the 
operational equipment. The simulator is, of course, precisely the 
equipment used for training, provided that means exist to repeat 
selected samples of air traffic without variation and to record and 
score various characteristics of the controller's performance. 

Simulators capable of measuring the performance of air traffic 
controllers have been developed by the FAA. A radar air traffic sim- 
ulator was developed at the Civil Aeromedical Research Institute 
(CARI) in 1965 that was useful for research on training, selection, 
controller proficiency, and workload as functions of the number and 
speeds of targets, the display design, and- ^the like. The NAFEC 
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simulator (1969) had greater track-handling caoabilities than the one 
at CARI, Situations of greater complexity could therefore be sim- 
ulated. The NAFEC simulator clearly demonstrated the feasibility of 
measuring the performance of air traffic controllers objectively and 
reliably. This work at NAFEC is still going on, and the current 
NAFEC Digital Simulation Facility (DSF) is improved over the sim- 
ulator used in the initial study. Among other improvements, the 
current DSF pi'ovides virtually real-time scoring. 

Thus, it has been shown that the performance of air traffic con- 
trollers can be measured in an objective way. It is necessary to use 
representative samples of air traffic carefully standardized for 
level of difficulty, i.e., in density, complexity, and potential con- 
flictions. The duration of the observations must be reasonably long, 
probably an average of at least two one-hour periods. Objective 
performance data are ne^dpd to evaluate the effectiveness of various 
types of training. They would also have groat value for the purposes 
of selection, establishing qualification standards, evaluating the 
proficiency of developmental an<] journeyman controllers, and determ- 
ining controller workloads at various levels of traffic, thereby 
contributing to the design of sectors. 

D. FINDINGS 

• Relevance. The present curriculum provides the information 
needed by controllers to do their work. However, questions 
about the amount of time given to various segments of train- 
ing, the order of non-radar training folJowed by radar train- 
ing, and the value of training on prototype sec^tors (e.g.. 
Tango) can be resolved best by an experimental approach 
based on objective performance measures. (Appendix F) 

• Methods used to establish competence . Objective tests used 
eor classroom or textbook subjects are thoroughly appropriate 
and relevant to the instructional material in the training 
program. However, determining the proficiency of performance 
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at the ATC console (i.e.? over-the-shoulder evaluations) is a 
subjective procedure which has been demons t ited experimentally 
to have limited reliability. The feasibility of objective 
performance measures has been demonstrated at NAFEC but such 
measures arc not: in use at present, (Appendix E) 

• Selection . FAA studies show that selection procedures could 
be improved by incorporating performance tests that measure 
behaviors required of controllers at work. Further improve- 
ments arc possible by assigning candidates to the en route, 
IFR) or VFR option on the basis of their test scores. Such 
improved selection procedures would be expected to reduce 
the ouml^pj? of candidate controllers who fail to complete 
their training, (Appendix D) 
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V. SIMULATION DEVICES FOR AIR TRAFFIC CONTROL TRAINING 



The capabilities and limitations of the state of the art of sim- 
ulation devices for air traffic control training are considered here. 
After a brief review of the history of simulators, their capabilities 
for training controllers are addressed. The training simulation capa- 
bilities of the automated systems, National Airspace System, Stage A 
(NAS-A), Automated Radar Terminal System (ARTS) III, and ARTS'lI, and 
the prospects for their use in training are then examined, and 
strengths and deficiencies are identified. The need for more system- 
atic planning for future simulation requirements is also discussed. 

The purpose of this analysis is to provide information needed to 
make decisions concerning the number, type, and location of training 
facilities and equipment. Appendix C provicJes a more extensive dis- 
cussion, 

A . HISTORY 

For this study, the simulators of interest are those that 
reproduce under controllable conditions air traffic situations likely 
to occur in actual practice and to do so in a fashion suitable for 
training. Simulation of air traffic for training has a long history, 
both for military applications and for air traffic control. It has 
been exploited for both training and research for the Airborne Com- 
bat Information Centers and Tactical Data Systems of the Navy, the 
radar stations and the Semi -Automatic Ground Environment (SAGE) System 
of the Air Force, and major national command and control systems, in- 
cluling those of the Joint Chiefs of Staff and the Strategic Air 
Command. It is used widely for air traffic control training in for- 
eign countries and for training military controllers in the United 
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states. Substantial simulation work has been done for the FAA, first 
at the Technical Development Center in Indianapolis and later at NAFEC 
in Atlantic City (Vickers, 1959, 1972). The simulators at these 
facilities have been used to investigate such problems as simultaneous 
dual approaches, combining of approach control facilities, traffic 
flow patteriis, and airport sector selection. And, as mentioned m 
the previous Chapter, they have been used in system performance in- 
ve stigations. 

B. CAPABILITIES 

Some of the obvious benefits in the use of simulators for air 
traffic control training are: 

• Thoy permit experiences with air traffic to bo arranged in 
an order of increasing complexity that is optimally useful 
for training purposes 

• They permit immediate review and assessment of each training 

experience . , 

• They provide as much repetition of any type of traffic prob- 
lem as is required to achieve mastery 

• They need not interfere with actual operations 

• They permit students to experience uncommon but important 
events or situations without having to wait f9r their occur- 
rence in rea] life 

. scheduling is flexible and can be tailored to the overall 
training program and for periods appropriate to the subject'^ 
importance 

• Safety is preserved 

• The duration of overall training time is reduced. ■> 
There is no serious debate about the usefulness of simulation 

for training air traffic controllers. The value of simulation equip 
ment for training purposes has been demonstrated beyond question m 
pilot training, air defense radar operation, malfunction detection 
for electronic equipment, sonar operation, and the like (e.g., pp. 
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96-90 )• It should be expected that simulators wouuLd also Ix: effective 
in training air traffic controllers, although statistical data to 
this effect have not been collected • 

ABOUT REALISM 

As the word implies, simulation is the process of representing a 
real task or event; by implication, the simulation is not a complete 
duplicate of the real thing, although parts of what is being repre- 
sented may be duplicated. Depending on the purpose to be served, the 
degree of simulation, the portions of the system to be included, and 
the fidelity of simulation should vary. For example, it has been 
found that high fidelity of simulation is not important when training 
a person to perform tasks with fixed procedures (Prophet, 1900, and 
Cox et al., 1965). Precise sensory cues are important in training 
for tasks which require precise motor skills, such as accurate feed- 
back (or "control feel") on aileron and rudder controls in aircraft 
simulators. Only that part of air traffic control that is a precise 
sensory -motor skill requires high fidelity in the simulation. On the 
other hand, if the critical skills are mostly in the areas of decision 
making and communication, completeness rather than precise realism 
of the display on the scope will probably be most significant. In the 
final analysis, the validity of a simulation has to be proven by 
research and experiment. 

D. CAPABILITY OF SIMULATORS IN OPERATIONAL SYSTEMS 

NAS-A and ARTS III were Initially delivered with a limited sim- 
ulation capability to permit equipment checkout, to facilitate main- 
tenance, and to be used for initial facility shakedown. In the 
realization that these simulation capabilities would also be useful 
for training, the staffs of two facilities (Houston ARTS III terminal 
and Washington NAS-A center) were tasked to develop "patches" to the 
operational programs that would permit flexible training at designated 
positions without interfering with operational positions . The results 



35 



are the Enhanced Target Generator (RTG) for the ARTS III and the 
Dynamic Simulator (DS) for NAS-A, and both have teen demonstrated/-- 
These simulators need not interfere in any way with the use of oper- 
ational positions, although there is a question about the capacity of 
the computer and display facilities to handle training and certain 
volumes of operational traffic at the same time. 

The ARTS II has no simulation capability in the present speci- 
fications. 

These simulations can provide roalistic representations of traf- 
fic for digital operations but not for broadband or analog radar. 
All "static" data available in the facility computer memories, as 
well as live operational traffic, can be presented at the simulation 
(i.e., training) positions. The "t>tatic" data include: sector 
boundaries, airport positions, navigation aid positions, holding pat- 
terns, airways, and weather contours (NAS-A only), all of which can 
be displayed optionally at any of the simulation positions. Essen- 
tially all of the operations required for operational control of 
aircraft can be simulated and outputs can also be generated for 
the support positions (data man and handoff man), a," needed. As 
part of a simulation problem, tapes of scheduled simulated traffic can 
be prepared and entered into the computer before the problem begins, 
and flight strips will be printed out at the appropriate simulated 
support position. Also, simulated traffic entered into the simulated 
areas by the "aircraft pilots" will be associated with the stored 
traffic data, and tracks will be started automatically on targets 
that pass the system association rules. All positions will be con- 
nected by the standar(3 level 300 communi (nations system and "pilots" 
will control simulated targets in accordance with the trainee con- 
troller's instructions; thus, the pilot-controller communications are 
also simulated. 

''The program for the computer display channel (CDC) of NAS-A was 
not completed as of October 1974. 
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The principal limitations of those simulations arc: (1) limited 
availability of positions and capacity in an operational facility; 
(2) the ability of "pilots" to generate and control adequate numbers 
of simulated targets; (3) inability to simulate adequately inter- 
actions with adjacent facilities; and (4) the lack of recording capa- 
bility designed for training purposes. In addition, the simulations 
have limited flexibility in that they can be interrupted but cannol 
be "backed up" or have parts repeated in the middle of a simulation 
problem to point out a control error or a deviation from control 
rules or standard practices. 

As mentioned above, neither the NAS-A nor the ARTS III simulation 
capabilities include broadband radar. As discussed in Appendix C, 
it is possible to u^e symbology to represent the analog return, but 
this results in an unrealistic display. This is clearly a limitation, 
particularly with ARTS III, sinc:e the analog display is an integral 
part oC the operational mode in that system. Since the developmental 
has ample opportunity to observe real displays, this lack of realism 
should not be a problem, but special efforts will be required to 
minimi/A? any negative transfer that might occur because of this lack. 
For future broadband simulation requirements, discussecJ below, the 
feasibility of add.ing a simple radar target generator to ARTS III 
might be explored. A number of companies (Raytheon, Ilydrosy stems, 
Litton, Sanders Associates, and others) have off-the-shelf broadband 
simulators in the v25,000 price range that could be used to augment 
the ETG capability. * 

The concern about capacity is that the simulation will overload 
the computer system, requiring it to be cut back (NAS-A) or taken off 
(ARTS III) so as not to interfere with ongoing operations. Table 3 
shows the track capacity of the NAS-A system at each center and the 
traffic in the peak hour of the peak day in 1973. More than half the 
centers have the capacity to handle such extreme traffic loads. 
Further, there is substantial variation of traffic load throughout 
the day. For example. Fig. 9 shows the load pattern by hour for Los 
Angeles during the peak air traffic day of 1973. Track capacity was 
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exceeded for less than 0 hours on this peak day. It is evident that 
ample opportunity is available Cor Gcihoduling simulation training 
in NAS-A centers. 



TABLC 3. TRAFFIC (PEAK [lOUR OF PEAK DAY, 1973) 
AND TRACK CAPACITY AT CENTERS 







9020 


1973 


PEAK 












VERSION 


IFR 


HOUR 


TRACK 






AUl nUKIZhU 




& DISPLAY 


(millions) 


OPERATIONS 


PA D A P 1 TV 

LArALII Y 


CCCTf\DC 

otLIUKb 


rVU S 


rnMToni 1 CDC 


ALBUQUERQUE 


A 


CDC 


0.91 


275 


300 


29 


46 


280 


ATLANTA 


D 


CDC 


1.44 


365 


400 


46 


54 


469 


BOSTON 


A 


CDC 


fl 07 

U. 71 




J\rJ 


27 


58 


415 


CHICAGO 


D/E 


DCC 


1.65 


410 


500 


38 


69 


566 


CLEVELAND 


D/E 


DCC 


1.73 


535 


400 


43 


65 


573 


DENVER 


A 


CDC 


0.64 


200 


300 


30 


44 


273 


FT. WORTH 


D/E 


DCC 


1.23 


430 


350 


39 


57 


338 


HOUSTON 


A 


CDC 


1.15 


600 


300 


38 


60 


424 


INDIANAPOLIS 


D 


CDC 


1.29 


340 


400 


33 


49 


450 


JACKSONVILLE 


A 


CDC 


1.08 


375 


400 


35 


57 


407 


KANSAS CITY 


D 


CDC 


0.99 


335 


400 


36 


52 


365 


LOS ANGELES 


D 


CDC 


1.05 


325 


250 


34 


48 


361 


MEMPHIS 


A 


CDC 


1.09 


394 


250 


30 


42 


348 


MIAMI 


A 


CDC 


1.05 


365 


350 


29 


50 


■293 


MINNEAPaiS 


A 


CDC 


0.91 


264 


250 


28 


41 


316 


NEW YORK CITY 


D/E 


DCC 


1.61 


421 


450 


37 


62 


576 


OAKLAND 


A 


CDC 


0.91 


278 


250 


29 ■ 


44 


346 


SALT LAKE CITY 


A 


CDC 


0.41 


105 


250 


18 


43 


183 


SEATTLE 


A 


CDC 


0.60 


163 


250 


16 


35 


185 


WASHINGTON. D.C. 


D/E 


DCC 


1.37 


400 


400 


34 


63 


482 


TOTAL 






22.07 






649 


1039 


7650 


AVERAGE 






1.10 ^ 






32.5 


56.0 


382.5' 



Sources: FAA Sizing CommiUee, FAA Air Traffic Activity. IDA Survey, FAA AAT-UO. 

Ill t • M U 



30 

er|c ^ 



500 



400 



fid 

D 

§ 300 



2 

O 

S 200 



100 































1 
































































































































































































a 


15 
- 11 


7 
7 




































































/ 








\ 










TR/ 


/ 


CAF 


ACI 


TY 



























































































































































































ol 

12 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 12 
M N M 

STANDARD TIME 
Derived From: FAA "Air Traffic Acriviry— 1973" 



FIGURE 9. Los Angeles Air Traffic on Peak Day, 1973 
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E. ACADEMY SIMULATION REQUIREMENTS 

Since the simulation capabilities of NAS-A and ARTS III appear 
to be adequate for facility training, at least for the near future, it 
follows that separate simulators dt field locations arc not required. 
The needs of the Academy are a different matter. Both the existing 
but obsolete Servonics equipment, which is expensive to operate, and 
the Sylvania modules should be replaced if there is to be any signif- 
icant radar training done at the Academy. Some modules could be 
scavenged for a successor simulation installation. If all radar 
training could be done in the field, no simulation capability would 
be needed at the Academy. However, as will be discussed in later 
sections, there are important cost savingn to be had from early screen- 
ing, and this can most readily ha done at the Academy. In fact, for 
the smaller IFR terminals centralized training is almost mandatory, 
because these terminals, with only a few controllers on duty at any 
one time, have ori]y limited training opportunities. 

If at least some IFR terminal training should be done at the 
Academy, a simulation capability is required. Installing the NAS-A 
or ARTS III simulation hardware necessary to operate in a simulation 
mode would be costly ($4 million to $5 million for NAS-A) and would 
result in a simulation lacking broadband capability. 

The long-range simulation requirements for the Academy should 
be considered in the overall context of system planning discussed in 
the next section, but for the short term it seems clear that a sim- 
ulator should be acquired as soon as possible. A number of questions, 
such as whether the simulator should be configured for both terminal 
and en route training, can be answered after alternative training 
locations are examined (see next Chapter). 

F. FUTURE SIMULATION REQUIREMENTS 

Training requirements seem to have played a negligible role in 
the specifications for NAS-A and ARTS III. Recognition of training 
requirements as legitimate design considerations could have resulted 
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in much more powerful simulation tools than were actually produced. 
Further, improvements that are feasible 5 such as realistic speed 
changes, are not being included in the rewrite of the dynamic sim- 
ulation program that is under way at the present time. It is im- 
portant or future system effectiveness that the need Cor training 
simulation be included in the design of future versions of the 
National Airspace System* 

A systems approach to simulation appears to be essential. Re- 
quirements for future simulation involving processes such as conflict 
detection, conflict resolution, and intermittent positive control can 
cause ATC disp.layed information to be quite different from what exists 
at present. Effective simulation of these processes can be difficult 
and expensive if it is done as an afterthought isolated from the main 
system development program. A centrali^ied group with knowledge of 
training needs, available ATC operational hardware arid software, and 
competence in minicomputer applications could provide the systems 
approach to applications of simulation in training which is needed to 
cope with future simulation requirements. 

G. FINDINGS 

• The NAS-A and ARTS III simulators appear sufficiently real- 
istic but have some limitations. Present and future defic- 
iencies, however, can be remedied by target -generating soft- 
ware and inexpensive hardware and recording capabilities. 

• These present simulators can be used at most scheduled times, 
particularly for proficiency, maintenance, and refresher and 
supplemental training. 

• A program of software for installation of performance measure- 
ment should be initiated. 

• A simulation capability for ARTS II equipment is warranted 
and needed now. The modular structure of ARTS II facilitates 
the introduction of this capability before the hardware is 
installed . 
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• As will be seen in later sections, a simulation capability 
is required at the Academy. 
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VI. ANALYSES OF ALTERNATIVE DEVELOPMENTAL 
TRAINING PROGRAMS 



The purpose of this Chapter is to identify and evaluate alter- 
native methods currently and potentially available to meet the train- 
ing needs of developmentals. The alternatives which are considered 
reflect differences in degree of centralization and differences in 
the duration of training, both issues of concern to the FAA- 

A. DESCRIPTION OF ALTERNATIVES 

The following options are significant in determining the possible 
restructuring of the FAA training system: 

• Location of training Academy or facility 

• ,Duration of training Related to curriculum (6-9 

mbntlis) or to necessary time for 
experience and associated Civil 
Service progression (3, 5-4, 5 
years) 

• Need for separate simu- Academy and/or facilities 
la tors 

• Responsibility for final Academy or facility, 
screening 

To analyze these options, it was assumed that five alternative 
training programs would be considered, each containing some combin- 
ation of the main features shown above. One of the alternatives to 
be considered is the present program. All are summarized in Table '4 
and explained below. 



^3 



TABLE 4 



DEVELOPMENTAL TRAINING ALTERNATIVES 
(GS-7 to GS-12/13) 



Alternatives 



Accelerated Extended Present Extended Accelerated 
Academy Academy Program Facility Facility 



Training Duration, 



6-9 7-10 7-10 7-10 6-9 



months 
Elapsed Time, 



0.5-0.75 



3V5-4.5 3.5-4.5 3.5-4.5 



0.5-0.75 



years 
Final Screening 
Separate Simulator 



Academy 

At Acad- 
emy 



Academy Facility Facility 

At Acad- No No 
emy 



Facility 

Terminal 
simulation , 
at Academy 



Some aspects of training are not at issue and, in the analysis 
that follows, it is assumed that these aspects are treated in the 
same way in all five altei^native programs. This applies, for example, 
to the contents of the curriculum, the order of presentation of topics, 
the amount of time devoted tc each phase of training, and the methods 
used to establish competence. The Academy would be responsible for 
developing lesson plans, training materials, and tests and for keep- 
ing them up to date. 



Common to each alternative would be the adoption of the improved 
entrance selection criteria discussed in Chapter IV. Revised pro- 
cedures and reordering of course materials to reflect present auto- 
mation and radar capabilities, improved pedagogical concepts, and 
performance measures would be incorporated as available in each alter- 
native. The advantages of simulators would be exploited throughout. 
The present Civil Service grade structures could be retained. Re- 
visions of job assignments and responsibilities would be required for 
several of the options examined. 

1. Accelerated Academy 

In this alternative, the student would receive all his academic 
training at a centralized facility such as the Academy at Oklahoma 





City. The training would be continuous and completed over a period 
estimatt:d to be from 6 to 9 months (as curriculum, measurements, and 
experience indicate) and would include simulation to prepare the 
student for final sector or position qualification at his home 
facility. A capability to simulate both en route and terminal facil- 
ities would be maintained at the Academy. Separate courses would 
be scheduled for controllers destined for en route, IFR, and VFR 
facilities. Screening would be done St the Academy and not take 
more than 9 months or so, and it would be done before the trainee 
achieved status as a civil servant. Entry into the ATS would not 
occur until after successful completion of Academy training, when 
status in a Civil Service grade would be assigned. The first sector 
or position qualification would normally take another several months 
at the home facility. Progression to fully qualified journeyman * 
controller would take place as the necessary experience and season- 
ing are gained. 

2. Extended Academy ' ' 

As above, all formal training would be given at the Academy, ^ ^ 
but the training for each phase would be followed by service and 
on-the-job training (OJT) at the home (or nearby) facility. The 
overall time would approximate the 3.5- to 4.5-year period of the 
present schedule and the present Civil Service ATC grade structure. 
Screening would be done at the Academy at the completion of each 
training phase. Time to reach final or position qualification for 
journeyman status would presumably be shorter than for the first 
alternative because of greater familiarity with facility operations. 

3. Present Program 

This is the present prescribed training program, in which the 
training load is shared by the Academy and the facilities. Screen- ' 
ing would be primarily done at each facility. 
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4. Exl'GfKlocI FaelJ il-y 

This alternative- {features the possibility of accomplishing all 
training at each taciiity (or in lo(-al groupings, where more appro- 
priate). The training plan and progression would be centrally devel- 
oped and standardized, aJ would the scheduling and performance of 
regular inspections. After completion of each phase of developmental^ 
training, facility service and OJT would follow at the facility. As 
in the "Extended Academy" option; the elapsed time to qualification 
would take 3.'j to 4.b years. Final qualification would be similar to 
present procedures, screening responsibilities resting with the facil- 
ities . 

5. Accelerated Facility 

This case is similar in pace to the "Accelerated Academy" option 
but is done in a continuous, decentralized fashion at centers and , 
terminals (or groups of terminals, as appropriate). This reflects 
the objective of completing formal academic training before proceed- 
ing to OJT and before Civil Service status is achieved. Again, course 
materials, plans, and final examinations would be centrally prepared^. 
Upon passing, the stiadent would be admitted to the ATS with the appro- 
priate Civil Service grade. The initial period would take approx- 
imately 6-9 months, as in the "Accelerated Academy" case. There- 
after, the student would commence to gain final sector and position 
training and experience, and ho would normally qualify in another 
few months. 

B. TRAINING SPECIFICATIONS FOR EACH ALTERNATIVE 

Underlying the five foregoing alternatives are a number of de- 
tailed specifications for the training programs for en route and 
terminal controllers. These include the time and location of each 
phase and subphase of the training, the mode of training (i.e., class- 
room/laboratory, OJT, simulation), student/staff ratios. Civil Ser- 
vice grade classification, and the elapsed time while in develop- 
mental status. As far .is possible, procedures and times that apply 
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to present training are used. For facilities, averages were derived 
from the results of the IDA survey of field facility training and 
then applied consistently to the several alternatives. All the de- 
tails are recorded in Appendix 11. As an example, Table 5 shows the ^ 
specifications for training of en route controllers j the levoi of 
detail is evident. 

C. TRAINING COSTS FOR EACH ALTERNATIVE 

An extensive effort has been made to develop cost estimates f or ^ \ 
each of the five alternatives. The costs of training are based on § 
the workload and practices in FY 1974. The cost data base underlying • 
all estimates derives from FAA budget submissions and data specifi- 
cally collected for this study from the FAA Academy and the en route' 
and terminal facilities. Many of these data are new. 

The principal measure used for comparison of alternatives is the 
variable training cost per year per one thousand graduates. Variable 
costs consist of v/agcs and benefits of instructional and support staffs 
student travel and per diem, and wages and benefits of students dur- 
ing their period of actual instruction. Capital costs, while not 
unimportant, approximate ten percent of these variable costs. All 
the details of the costing process are contained in Appendix G and 
are an important part of the comparison. 

Table 6 shows the costs for each alternative way of training 
for each controller option. 

The cost differences among alternatives are caused by a complex 
set of differences in the specification of each training program of J 
the sort illustrated in Table b. However, one major difference be-, 
tween the costs of Academy training and facility training is the per l 
diem and travel costs of the former. The per diem cost could be re- 
duced with residential facilities operated by the Academy. More 
importantly, however, "accelerated" training costs could be more than 
compensated by savings from possible reduced training time. These 
and similar considerations suggest that with present knowledge only 
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TABLE 6. VARIABLE TRAINING COSTS PER THOUSAND GRADUATES 

(dollars in millions) 

Terminal 



Alternative 


En Route 


IFR 


VFR 


Accelerated Academy 


$33.6 


$25.8 


$10.6 


Extended Academy 


32.3 


29.5 


15.7 


Current Program 


29.9 


24.2 


12.8 S 


Extended Facility 


29.4 


22.9 


12.5 


Accelerated Facility 


27.2 


18.7 


8.2 



minor cost differences can be assuredly distinguished among alter- 
natives. 

At current rates, about 900 en route controllers are hired each 
year. Thus, the overall training costs each y3ar greatly exceed the 
one-time cost of simulation necessary for the facilities and for the ^ 
Academy. 

D. TRAINEE SALARIES 

The cost estimates in Table 6 include the fu"! salaries of train- 
ees while actually engaged in training. In Alternatives 2, 3, and 4 
the trainee spends a considerable portion of his developmental period 
performing at his facility those sup'^orting tasks for which he is 
qualified. Since these opportunities are limited, some portion of 
his salary should be considered as a training cost, yet just what 
frac.tion should be assessed against salary is uncertain. Table 7 

sr 

shows the variable costs when 0, 25, 50, 7b, and 100 percent of the 
nontraining time salaries are included as a training cost for the 
en route, IFR, and VFR options. 

The data in the table show the effect that stretching out the 
training period has on cost. In fact, a major cost factor is the 
duration of training. 

Because of such considerations, the cost implications of selec- 
tion procedures that are inferior to those now available and of late 
recognition of the inability of a developmental to be a capable 
journeyman controller have been determined in this study. Also, the 

49 



ERLC 



58 



additional costs introduced by the great fluctuations in student 
enrollments at the Academy have been estimated. The details of those 
costs are given in Appendix G, 

TABLE 7 VARIABLE TRAINING COSTS OF ALTERNATIVE WAYS OF 
TRAINING AIR TRAFFIC CONTROLLERS 

(dolJars in mil] 



Option & Alternative 

En^outc Centers 
Accelerated Academy 
Extended Academy 
Present Program 
Extended Facility 
Accelerated Facility 

IFR Terminals 

Accelerated Academy 
Extended Academy 
Present Program 
Extended Facility 
Accelerated Facility 

VF^ -Terminals 

Accelerated Academy 
Extended Academy 
Present Program 
Extended Facility 
Accelerated Facility 



ns por 


1000 controllers 


era inod ) 




Porcentago 
Salarit-'s 


of Nont: raining Time 
Charged to Training 




0% 


25% 


50% 


75% 


100% 


$^3.6 










32.3 


$43.4 


$54.5 


$65.6 


$76.7 


29.9 


41.3 


52.6 


64.0 


75.4 


29.4 


41.1 


52.9 


64.6 


76.3 


27.2 
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29.5 


39.9 


50.2 


60.6 


70.9 


24.2 


34.3 


44.3 


54.4 


64.5 


22.9 


33.0 


43.1 


53.1 


63.2 


18.7 
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15. 


7 


22.9 


30.2 


37.4 


44.7 


12 


8 


19.8 


26.9 


33.9 


41.0 


12 


.5 


19.6 


26.7 


33.7 


40.7 


8 


.2 











Accelerated training and consequent earlier screening could re- 
duce the cost of training as much as $42, 000-$48, 000 for an en route 
controller, $39, 000-$46, 000 for an IFR terminal controller, and 
$30,000-$33,000 for a controller at a VFR terminal facility. 
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Improved selection procedures to reduce the losses of develop- 
mentals during training could reduce training costs a maximum of 33 
percent for en route controllers and a maximum of 22 percent for 
terminal controllers. These figures represent a perfect selection 
process and are unachievable. A 50 percent reduction of trainee los- 
ses, however 5 would provide cost reductions of 16 percent in en route 
training and 10 percent in terminal training. 

Smoothing out the flow of students at the FAA Academy and staf- 
fing the Academy to deal with a steady training load could produce a 
saving of between 20 and 30 percent of the cost of Academy controller 
training operations. This would have meant something between $0.9 
million and $1.4 million in FY 1974. 

E. FINDINGS 

• Within the accuracy of cost estimation, the cost differences 
between training at the FAA Academy and at the field facil- 
ities are small, and their significance is masked by uncer- 
tainties associated with the details of the various alter- 
native training programs. 

u » 

• Smoothing the enrollments of developnrental controllers at 
the Academy could result in cost benefits of approximately 
$1 million each year, in addition to improved training per- 
formance. 

• An 11 to 16 percent cost reduction could result from selec- 
tion procedures that reduce trainee attrition by 50 percent. 

• A major cost factor is the dui^ation of training. Accelerated 
training and resultant earlier screening could reduce train- 
ing costs as much as $48,000 per developmental. 

• Simulation costs, whether at a facility or at the Academy, 
are small compared to the training costs. 
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VII. IMPLICATIONS 

The purpose of this Chapter is to consider the significance to 
the FAA of the findings of this study and to develop some suggestions 
for action. Attention is given to (a) simulators and centralized or 
noncentralized training, (b) the training budget, (c) future require- 
ments for training, (d) R&D for training, (e) hiring practices, (f ) 
standardizaltiori, and (g) the training load at the Academy. 

A. SIMULATORS AND CENTRALIZED OR NONCENTRALIZED TRAINING 

Since the Corson Report (Corson et al., 1970), the FAA has been 
concerned with how to implement recommendations that simulation equip- 
ment should be procured for training en route and IFR terminal con- 
trollers at a centralized location. The findings of this study bear 
directly on this problem. As presenif-d in the previous Chapter, the 
major part of training costs is the variable cost (i.e., that due to 
wages and benefits of instructors and students, travel and per diem). 
This cost is about $24,000 to $30,000 per .qualified developmental, 
depending on controller option. In addition, a charge should be 
added to reflect the nontraining time salary while not a fully quali- 
fied journeyman. 

To train 1000 controllers costs from $24 million to $30 million. 
A full-fledged simulator for a central location would cost less than 
$10 million and would be available to more than 10,000 controllers 
over 5 years; thus, it would cost less than $1000 peT con^roiler. 
Simulator costs are therefore small compared to actual controller 
training costs. 
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There are not significant cost differencos between centralized 
and decentralized training modes— between training at the Academy and 
training at a facility. 

Major cost factors are introduced by the duration of training and 
by any delay in screening. These can account for more than half the 
training cost to produce a qualified controller. Centralized train- 
ing has inherent advantages in these cost areas. A centralized train- 
ing facility can train a student full time, a capability possible at 
only the largest of operating facilities. Thus, screening can take 
place most quickly. Standardization of training, measurement of 
training effectiveness, and application of uniform screening procedures 
can all be done easily and objectively at a central facility. Ob- 
viously, such a centralized facility must be fully equipped to accom- 
plish its mission, and this would include an air traffic conti>ol sim- 
ulator fitted for training. This should be provided in the very near 
future. 

Proficiency, remedial, supplemental, and other testing and 
training can be done more easily at a facility, particularly with 
central support. The simulated training and measurement capabilities 
possible in the NAS-A, ARTS III, and ARTS II equipment is important 
in this aspect of the controller training program. Some capability 
for training almost exists in the exi'^i-ing NAS-A and. ARTS lit equip- 
ment. As an interim measure, the "patches" required to make those 
equipments useful for training can and should be extended immediately 
to the entire system. These "patches," of course, are the Dynamic 
Simulator (DS) for NAS-A, demonstrated at the Leesburg Center, and 
the Enhanced Target Generator (ETG) for ARTS III, demonstrated at the 
Houston Terminal. Provision should also be made soon for "pilot" 
consoles to conserve PVDs," and for recording and playback equipment 
to permit the rpwiew of performance so important for training. All 
of these steps should be regarded as readily feasible, near-term • 
steps to realize the current almost-present capability for training. 
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Dedicated simulators for training might well be needed in the 
longer term, although a strong case cannot be made for them at the 
present time. The long-term case rests upon (1) the growth of traf- 
fic and the addition of new control features which might reduce the 
availability of the NAS-A and ARTS for training,. (2) the specif ioat.ion 
of required performance measures an^i recording .and analytical capabil- 
ity to provide training scores (which depends on work still under way 
at NAFEC), and (3) management arrangements to ensure adherence among 
centers and terminals to the curriculum and standards promulgated by 
the national training program. Planning fur the characterization of 
the required, simulators should start immediately. The "patches" sug- 
gested above as feasible and relatively inexpensive in the near term 
cannot be regarded as an ideal solution. The simulators in 'NAS-A and 
ARTS III were designed for purposes of maintenance, but they can and 
should be adapted now to training as well. When this is done, there 
will still be a need for more competent and flexible simulation capa- 
bilities for training, and steps to specify and install these, over 
the next 3 to 5- years, should also be started immediately. 

B. THE TRAINING BUDGET 

The sources of training funds and their control are spread among 
a number of offices within the FAA.. As a result, it is difficult to 
identify all the points in the budget which relate to training and 
hence to identify the full costs of training. Of more importance, 
the dispersion and diffuse control of training funds in the present 
budget structure makes it difficult, if not impossible, to administer 
a uniform national training system. Appendix I discusses the budget- 
ary sources of controller training and coincidentally derives an 
estimate of ATC training cost— about 5 to 8 percent of FAA operating 
expenses. It also discusses problems inherent in the present method 
of funding training activities with respect to coherence of the pro- 
gram and standardization of its content and quality. 
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Funds are allocated from the FAA appropriation to organizational 
offices and institutions (i.e., the Aeronautical Center, Regions, 
etc.) without binding functional application. For example, funds to 
the Regions, budgeted for training, travel, and per diem can be — and 
often are — used for other operating functions at the discretion of 
field administrators. Variations among facilities in the conduct of 
the training program, as reported in this study, are probably related 
to this method of budgeting. Consequently, planning for training 
and control of its application becomes almost impossible, and utili- 
zation of training resources becomes highly variable. 

Further, sizable amounts of money are involved in the air traf- 
fic controller training system. Table 8 shows the FY 1974 funds 
related explicitly to controller training. The total approaches 7 
percent of the FAA operating accounts, increasing to 10 percent if 
developmental nontraining time salaj^ies are included. Centralization 
of fiscal authority for controller training activities does not exist 
at present. If it is warranted, as is probably the case, a plan for 
a new training budget structure, together with organizational re- 
sponsibilities, will have to be developed. 



TABLE 0. 



AIR TRAFFIC TRAINING COSTS, FY 1974 
(dollars in millions) 



Centralized Training Budget Costs 



$18.9 



Allocation of Centralized Training Support & Travel $8.2 
Allocation of Management Training School (MTS) 4.1 
Air Traffic Branch (ATB) 6.6 



Field Facility Training Staffs 



13.1 



Trainee Salaries 



46.7 



TOTAL 



$78.7 



^Includes periodic training of full-performance controllers 
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C. FUTURE REQUIREMENTS 

The "Ten Year Plan, 1973-1982" (FAA, 1973a) and FAA Engineering 
and Development Programs (FAA, 1974b) project many improvements to 
the National Airspace Traffic Control System. Many of these will 
have significant implications for training and for new features not 
present in any training equipment contemplated so far. The follow- 
ing operational improvements must be considered in this category: 

1. Air Route Traffic Control Center (ARTCC) 

a. Conflict detection 

b. Conflict resolution 

c. Intermittent positive control (IPC) 

d. Discrete address (tactical) data link of discrete address 
beacon system (DABS) 

e. DABS interrogation hierarchy control. 

2. T(.->rminal Radar Approach Control (TRACON) 

a. Automatic radar level tracking with tagged targets 

b. Metering and spacing 

c. Conflict prediction 

d. Conflict resolution 

e. Fail safe/Fail soft/Auto standby switch 

f . Multiradar • 

g. Multiprocessing 

h. DABS data link. 

3. Tower Radar Automation Cab (TRACAB) 

a. Radar level tracking 

b. Noise abatement pattern control 

c. Wake turbulence separation control 

d. Multisegmented approach and departure routings to 
exploit the characteristics of area navigation (RNAV) 
and microwave landing system (MLS) 

e. Upgrading ARTS II from beacon data level to beacon 
tracking level. 
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Attention is directed to the importance of including the train- 
ing implications of these improvements early in the development cycle. 

D. R&D FOR TRAINING 

This section presents several research projects that would 
support further improvements in the selection, performance measures, 
and training of air traffic controllers. The FAA has supported and 
is supporting research that has direct application to training. How- 
ever, the current level of support is so modest that it will take a 
long time before the needed results become available. Therefore, 
several research areas have been identified in this study as worthy 
of immediate attention: 

• Selection . Little is known about how the criteria used dur- 
ing selection affect assignment to controller option or the 
quality of performance after completion of training. Im- 
proved selection procedures would be expected to reduce 
attrition during training and thereby save some of the ex- 
penses due to attrition (Appendix D). 

• Performance Measures . These are very important to the FAA 
because, together with other factors such as cost and flexi- 
bility, -hey provide the means needed to evaluate the ulti- 
mate effectiveness of alternative methods of training air 
traffic controllers. Performance measures are also needed 
for many other purposes of interest to the FAA, such as 
determining, for example, the controller's maximum useful 
workload, the distribution of traffic loads among sectors, 
the impact of new or proposed types of ATC equipment, and 
the significance of various tests and criteria for the selec- 
tion of controllers (Appendix E). Such research should be 
focused on objective performance measures that could be 
employed in conjunction with the automated equipment at 
centers and terminals. 
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• Training. The present national programs for air traffic con- 
troller training specify in great detail the information and 
procedures that are to be taught during each phase of train- 
ing and the goals to be accomplished before the developmental 
can move from one phase to the next. Yet it is evident that 
there are substantial variations in the training programs at 
the many facilities. Substantial questions are outstanding 
about the content and method of training. Experiments de- 
signed to evaluate the significance of different ways of 
training are warranted (Appendix F). 

Research and development related to selection and training are 
conducted on behalf of the FAA, variously, by CAMI, the Academy, 
NAFEC, the Office of Aviation Medicine, the Office of Personnel and 
Training, the Systems Research and Development Service, and the Air 
Traffic Service. Although there is a clear interdependence between 
many of these efforts, there are no ready moans for coordinating the 
various perograms. It is suggested that a focal point be established 
to coordinate these research efforts and to respond to findings. 

4 

Initially, an Advisory Committee on Personnel Research and Training, 
reporting to the FAA Administrator, could be established for a trial 
period. 

E. HIRING PRACTICES 

The history of ATC hiring is shown in Fig. 10. The present 
practice is to delegate hiring to the Regions and Liie facilities they 
control. As a result, there is little or no synchronization of hir- 
ing across the country. Most hiring now takes place during the 
fourth quarter, when availability of funds is more certain and fiscal, 
year expenditures are thereby less, without the loss of the author- 
ization for controller positions. This results in large fluctuations 
in input to the training system and resultant saturation of the sys- 
tem for extended periods. There dre further implications for sched- 
uling and sequencing of the units of training. And, as already 
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Source: FaA RI5 PT 3300-5 



FIGURE 10. ATC Quarterly Hires, FY 1969-1973 
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discussed and as showx in Appendix G, this practice leads to in- 
efficient use of training resources. 

There are a number of standard management procedures that would 
resolve this problem without disturbing the decentralized operations 
of the National Airspace Systen.. In this study, the important de- 
tails of the hiring problem have not been examined, but it is very 
evident that improved planning and execution of the recruitment effort 
is needed. This, also, was a finding of the Corson Report (Corson et 
al., 1970). 

F. STANDARDIZATION 

As mentioned before, there is substantial evidence of lack of 
standardization in training, use of training resources, and operating 
practices. 

It might be thought that standardizacion could be achieved by 
vesting the Academy with sole jurisdiction over <)11 training, or at 
least over portions of the training program to be standardized. The 
Academy has had exclusive responsibility for portions of the train- 
ing program in the past, but no such arrangement has survived for an 
appreciable length of time or over large changes in the numbers to 
be trained. Other requirements for a viable centralized training 
program have not been met in the ^^ast. One of these requirements 
has to do with budgeting, as discussed above. Another has to do 
with hiring, also discussed above. Standardized training requires 
central »:jnd systematic hiring on the basis of projected needs. 

Finally, standardization of training without centralization 
could be achieved, but the requirement for centralization of other 
functions remains, and some office must be vested with a positive 
and absolute role of certification of both field facility training 
programs and the qualifications of individual trainees. In some way, 
the independence and responsibility cf this certifying office would 
have to be ensured. 
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G. THE TRAINING LOAD AT THE ACADEMY 

Figure 11 shows the number of residents in air traffic control- 
ler courses at 'the Academy for the period FY 1968-1974. Large 
fluctuations are evident, but, in this case, they are not cyclical 
and thus could not be anticipated long in advance. Over this period 
the Academy staff size has remained relatively stable. Thus the 
$,taff at the Academy is generally either overloaded or overmanned 
and is only occasionally in proper balance with the student load. As 
a practical matter, development of new course materials and surveys 
of training efi ectivencss are set aside at the Academy when there is 
a large influx of students. Since the, dominant cost at the Academy 
is maintaining instructional staff, unused capacity implies high 
unit cost and inefficiency, as discussed in Chapter VI and Appendix 
G. 

It is estimated that the cost of centralized training can be 
significantly reduced by smoothing the flow of training. The re- 
sultant savings in FY 1974 could have been about $1 million. 

H. FINDINGS 

• Centralization . The full-time training done at a central- 
ized facility permits the earliest screening of develop- 

' mentals and the fastest development of productive controller 

capabilities. Great cost benefits accrue to such a training 
procedure as well as the desirable application of uniform 
training procedures, performance measurement, and screening. 
All the full-time training necessary to ready a developmental 
for qualification should be done at a centralized facility. 
Exceptions could be large facilities where the developmental 
could receive full-time training. Qualification and season- 
ing through experience would be done at the home facility. 

• Simulations . The centralized facility, presumably the 
Academy, should be equipped with a fully capable air traffic 
control simulator designed specifically for training. NAS-A 
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and ARTS III simulation^^can serve training and proficiency 
testing where NAS-A and'^AfeTS III are installed. The training 
limitations of these equipments should be upgraded as quickly 
as possible. Simulation capabilities should be introduced 
in the new ARTS II equipment currently under procurement. 

• ATC Training Budget . A system to ensure that budgeted funds 
are used for controller training is indicated. These funds 
account for 7 to 10 percent of FAA operating accounts. 

• Future Requirements . The training and simulation implications 
of the many improvements to the National Airspace Traffic 
Control System should be identified early in the development 
cycle. This is important in order to procure hardware and 
software and to develop proper training programs in a timely 
fashion. 

• R&D for Training . Further research should be undertaken in 
controller selection, controller performance measures, and 
training alternatives. Improved coordination of and respon- 
siveness to all personnel-related R&D, including ongoing 
research, is required. 

• Hiring . A mechanism to smooth recruiting and hiring should 
be adopted. 

• Standardization . There is substantial evidence of lack of 
standardization in training. An FAA office should be vested 
with a positive and absolute role of certification of both 
field facility training programs and the qualification of 
individual trainees. 

• Academy Training Load . Substantial cost benefits (about §1 
million in FY 1974) and efficiencies could result from smooth- 
ing the flow of training. 
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APPEITOIX A 
STATEMENT OF WORK 



To successf "ly accomplish this requirement the Contractor shall 
identify and evaluate alternative ways of improving the air traffic 
controller training system. Alternative methods of training will be 
evaluated in terms of (a) system costs, (b) length of training time, 
(c) system capacity, (d) flexibility to accommodate to changes in 
training loads, and (e) flexibility for incorporating new training 
specifications . 

^ Tasks , The following tasks will be undertaken in an effort to 
identify and resolve the main issues which influence the training of 
en route and terminal air traffic specialists: 

Task 1. Review the evolution of the existing national programs 
for the qualification, refresher, proficiency maint- 
enance 5 and supplementary training of en route and 
terminal control specialists. Develop estimates of the 
number of individuals who will require various types of 
qualification and supplementary training, identify 
alternative methods of accomplishing the required 
training, and establish the questions which must be 
ansV7ered in order to evaluate the various ways in 
which training needs could be satisfied. 

Task 2, Assess the adequacy of current specifications for train- 
ing. Compare (1) the contents and relevance of present 
training courses and (2) the methods used to establish 
competence at their completion to the current specific- 
ations for training. Examine the extent to which cur- 
rent specifications for training are based upon analyses 
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of on-the-job performance -and the degree to which quali- 
fication standards have been validated against operational 
performance data. Included would be an analysis of the 
need for, and a description of, the requirements for 
standardization and quality control in terms of the im- 
plication or impact of the lack thereof on training. 

Task 3. Appraise the capabilities and limitations of the state- 
of-the-art of simulation devices relevant to air traffic 
control training. 

Task 4. Establish, for the purpose of analysis, the alternative 
methods currently and potentially available for train- 
ing en route and terminal control specialists. This 
shall, in general, describe the applicable simulation, 
on-line, and classroom facilities that nay be required 
for training the anticipated loads. The analysis shall 
examine the influence of such factors as the number, 
type, and complexity of equipment, location of the train- 
facilities, and the sequence of progression through 
training blocks upon the number of individuals who may 
be trained per unit time, the duration of training, and 
the total annual cost of the training program. The pur- 
pose of the analysis is to provide information needed to 
make decisions concerning the number, type, and location 
* of training facilities and equipment. 
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FIELD FACILITY SURVEY ON TRAINING 
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A. INTRODUCTION 

The purpose of this Appendix is to describe a survey that was 
performed to determine the numerical characteristics of the air 
traffic controller training performed at facilities during FY 1974. 
The survey was undertaken after it was found that little was known 
about the overall effort that is directed to developmental and 
proficiency training. The information solicited in the survey was 
needed for a variety of purposes, a few of which are: determining 
temporary training assignments and training losses; determining the 
costs of training for FY 1974; and determining the timing of losses. 

A copy of the questionnaire used in the survey and a sample of 
the results are presented in this-^ Appendix. 

It should be emphasized that the survey was concerned with 
training activities during FY 1974. It offers a snapshot of the 
situation prevailing at that time. Applications to other periods 
should obviously be made with. care. The results, however, may be 
useful to activities other than training. 

B. FIELD FACILITY SURVEY QUESTIONNAIRE 

The survey was intended to cover every facility performing air 
traffic control. This included air route traffic control centers 
(ARTCCs), airport traffic control towers, combined stations/towers, 
common IFR room RAPCONs/RATCCs , RAPCONs/ATCCs , and RAPCONs/ARTCTs 
The addresses were obtained from the FAA's National Field Office 
Directory . All the ARTCCs submitted responses, but a few question- 
naires for terminal facilities were returned because of improper 
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address* Most of the terminals responded. Overall, 78 percent of 
IFR terminals and 70 percent of VFR terminals responded. The results, 
therefore, are an extensive sample of FAA air traffic control facil- 
ities. The support of these facilities is gratefully acknowledged. 
Filling out the questionnaire involved substantial effort, as the 
data requested were of a kind not normally collected about training 
operations. More information on the results is given in Section III 
of Appendix G. 

A copy of the survey questionnaire follows. Essentially the 
same form was sent to both ARTCCs and terminal facilities; the sole 
difference lay in the designations of sequential training phases. 
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FIELD FACILITY TRAINING SURVEY 
(LEVEL & CHARACTERISTICS OF TRAINING OPERATIONS) 



Type of Facility (1). 



(En Route = 1 ) 

(VFR Tower = 2) 

(IFR Tower (Non-radar) = 3) 

(Tower/TRACON = 4) 

(TRACON = 5) 



Facility Sequence Number (2)^ 



Region Number V (7)_ 



Operations Level Classification (8)_ 



Automated Traffic Control Equipment - Scheduled 
or In-place 



Key: En Route: 



CDC 
DCC 



= 1 

= 2 



Tower/Jerminal ; ARTS III = 3 

ARTS II =4 

Radar Only = 5 

Non-Radar = 6 



No^e; Pootnott6 aKt ioand at tht end oi qanttionnalKQ.. 
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I. CONTROLLER COMPLEMENT AS OF YEA-R-END, FY 1974 



10 



(01) 



(02) 



(03) 



A. 
B. 



II. 



Authorized Controllers 

Other 2152 Series Required to 
Maintain Facility Currency 

Assigned li On-Board 

Full Performance 

Pre*Developmental 



Number 

(ii)_ 



Phase I 

Phase II, Non-Radar 
Phase II, Radar 
Phase III, Non-Radar 
Phase III, Radar 
Assigned ( On Detail to FAA Academy 
Pre-Developmental 
Phase II, Non-Radar 
Phase II, Radar 



(16)_ 

(21). 
(31). 
(41). 
(51). 
(11). 
(21). 



(31 )_ 



Average GS 
Grade Level 



(26)_ 
(36)_ 
(46)_ 
(56)_ 
(16)_ 
(26). 
(36). 

(41). 
(46). 
(51). 
(56). 



NUMBER OF DEVELOPMENTALS AND PRE-DEVELOPMENTALS 
DETAILED TO FAA ACADEMY DURING FY 74 



A. Pre-Developmentals 

B. Developmentals; 

Phase II, Non-radar 

Phase II, Radar 



(11). 

(16). 
(21). 
(26). 



MI. NEW HIRE DEVELOPMENTALS REPORTING ON-BOARD DURING 
FY 1974, BY QUARTER 1/ 



/// 



/// 



1st Quarter 

2nd Quarter 

3rd Quarter 

4th Quarter 



New To 
FAA 



(31). 
(41). 
(51). 
(61). 



Transfers From 
ARTCC i FSS_ 



(36). 
(46). 
(56). 
(66). 



/// 
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IV. TRAINING PERSONNEL COMPLEMENT 



A. Average Full Time Staff During FY 1974 ^ 

(EPDOfEPDS) (11) 

B. Part Time or Temporary Detail 
Personnel; "Full Time 
Equivalent" V 

Man-Months During FY 1974 (16) 



V. NUMBER OF CLASSROOMS/LABORATORIES 

MAINTAINED BY FACILITY (21) 



VI. NUMBERS OF DEVELOPMENTALS AND PRE-DEVELOPMENTALS 
LEAVING TRAINING PROGRAM DURING FY 1974. BY PHASE 

While Failing While Progressing 
Program Satisfactorily 



(05) Pre-Developmentals (11) (16) 

Phase I (21) (26) 

Phase II, Non-Radar (31) (36) 

Phase II. Radar (41) (46) 

Phase III. Non-Radar (51) (56) 

Phase III. Radar (61) (66) /// 
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FOOTNOTES 



Region number key: 



Eastern 
New England 
Southern 
Great Lakes 
Central 
Southwest 
Rocky Mountain 
Northwest 
Western 



(EA) = 1 




(GL) = 4 




(RM) = 7 
(NW) = 8 
(WE) = 9 



Operations level classification key: 



ARTCC level 2 



ARTCC level 1 



Terminal /tower level 1 

Terminal/tower level 2 

Terminal /tower level 3 

Terminal /tower level 4 



= 1 
= 2 
= 3 
« 4 
* 5 
= 6 



Include only personnel new to the ATS. 

A full time equivalent man-month is the number of hours per month worked 
by a full time employee. A specialist devoting an average of one-half 
time to training would be counted at a rate of one-half full time 
equivalent man-months per month. 

Persons initially entering combinations of classroom, self-study, 
OJT, and other than training activities during the year should be 
counted once in each applicable category. For example, a person who 
first enters phase III non-radar classroom, phase III non-radar OJT, 
phase III radar classroom, and who is promoted to GS 11 during FY 
1974 would be included in each of the counts for phase III non-radar 
classroom, phase III radar classroom, phase III non-radar OJT, and 
GS 11 other than training. He would not, however, be included in counts 
for those categories to which he had been assigned prior to the first 
day of FY 1974, e.g., phase II training or GS 9. 

Include only on-duty (paid) time. 

Include time spent in qualification check-out. 

Exclude FAM trips. 
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C. SOME SURVEY RESULTS 

This section presents some results of the survey coupled with 
some statistical data from FAA Air Traffic Activity publications 
(e.g., FAA, 1974a). More of the survey data, particularly those 
relating to costs of training, are presented in Appendix G, partic- 
ularly Tables G-13, G-14 and G-15. 

Manning and training at each center are obviously related. 
Each center has a number of sectors, and the number of personnel 
assigned to a center depends on the volume and complexity of traffic 
in the various sectors. This feature is examined herewith because 
of its significance to future training loads. 

Table B-1 shows some ratios between the numbers of controllers 
and indices of traffic load at 20 en route centers listed in de- 
creasing order of traffic volume (as measured by annual IFR aircraft 
handled in 1973). 

These ratios are plotted in Fig. B-1 against traffic volume, 
with a least-squares line fitted for each to assist in appraising 
the trend. From both the table and the plot, it is evident that 
each center has a unique set of operational characteristics, pre- 
sumably deriving from its own peculiar situation. 

The number of trainees at each center was also examined, based 
on data derived from the IDA survey. Table B-2 shows that there are 
very wide differences in the distribution of trainees according to 
their phase at each center. 
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TABLE B-2. NUMBER OF TRAINEES/DEVELOPMENTALS 
BY TRAINING PHASE AT 20 ARTCCs 



Phase 





PRE 


I 


11 NK 


TT D 


TTT MP 
J. J. J. Vis\ 


TTT R 


Tota3 


Cleveland 


0 


28 


3 


3 


38 


24 


96 


Chicago 


5- 


"132 


30 


0 


69 


18 . 


2S4 


New York 


0- 


93 


10 


21 


101 


. 30 


255 


Atlanta 


12 


96 


28 


0 


28 


55 


219 


Washington, D.C. 


4 


13 


28 


0 


49 


64 


158 


Indianapolis 


0 


68 


0 


7 


14 


37 


126 


Fort Worth 


15 


48 


3 


0 


8 


33 


107 


Houston 


4 


65 


0 


1 


0 


24 


94 


Memphis 


3 


86 


0 


0 


14 


2 


105 


Jacksonville 


6 


50 


87 


0 


9 


6 


158 


Miami 


S 


24 


24 


0 


1 


10 


67 


Los Angeles 


0 


8 


35 


12 


19 


60 


134 


Kansas City 


9 


57 


0 


0 


10 


17 


93 


Boston 


20 


0 


0 


0 


7 


57 


84 


Oakland 


15 


26 


25"^- 


0 


1 


34 


101 


Albuquerque 


14 


29 


8 


8 


3 


0 


62 


Minneapolis 


0 


64 


10 


. 0 


11 


8 


93 


Denver 


0 


8 


16/ 


5 


9 


10 


43 


Seattle 


12 


15 


i 


0 


2 


3 


38 


Salt Lake City 


10 


5 


0 


_0 


15 


1 


31 


Total 


137 


910 


308 


57 


409 


493 


2522 



NOTE: R = radar; NR = non -radar. 
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An associated consideration concerns the fraction of controllers 
who are fully qualified and the instructional resources at each 
center* Table B-3 shows the relationship. The fraction of fully 
qualified controllers ranges from 54 percent at Chicago to 85 per- 
cent at Denver. The relationship of instructional capabilities to 
apparent requirements is not obvious, (In fact, there seem to be 
moie training resources where there are fewer developmentals to 
train.) It seems inadequate, for example, at Chicago. The relation- 
ship between "student/teacher" ratio to fraction of qualified con- 
trollers is shown in Figure 

TABLE B-3. INSTRUCTIONAL RESOURCES, HIRES, AND ATTRITION . 
DURING TRAINING AT 20 ARTCCs 



Percentage of 

Operational Controllers Developmental 
that are Developmentals Losses 



En Route Center 


Fully Qualified 


per Instructor 


Hires 


Number 


Percenta 


Cleveland 


83% 


6.9 


56 


11 


11% 


Chicago 


54 


9.4 


109 


24 


9 


New York 


56 


12.4 


98 


36 


14 


Atlanta 


50 


20.7 


111 


12 


5 


Washington 


69 


8.3 


4 


6 


4 


Indianapolis 


72 


8.0 


45 


16 


13 


Fort Worth 


70 


7.7 


31 ' 


13 


12 


Houston 


75 


4.V 


61 


10 


10 


Memphis 


67 


17.0 


69 


13 


12 


Jacksonville 


64 


13.9 


102 


23 


15 


Miami 


72 


5.4 


49 


24 


36 


Los Angeles 


, 60 


12.2 


25 


18 


13 


Kansas City 


77 


6.9 


21 


9 


10 


Boston 


81 


3.3 


20 


14 


17 


Oakland 


70 


6.2 


29 


21 


21 


Albuquerque 


77 


3.0 


76 


17 


27 


Minneapolis 


69 


8.9 


39 


17 


18 


Denver 


85 


2.9 


38 


11 


26' 


Seattle 


81 


3.6 


0 


7 


18 


Salt Lake City 


83 


2.3 


29 


24 


77 



Source: IDA Survey* 
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PERCENTAGE OF OPERATIONAL CONTROUERS THAT ARE RULLY QUALIFIED 

FIGURE B-2. Rslaiion Ebtween Instructional Resources at Centers and Percentage 
of Controllers that Are Fully Qualified 
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In addition, the data show wide variations in training at term- 
inals also. Table B-4 shows data for some of the major IFR terminals, 
where the average number of developmentals per instructor is 7,3; 
this ratio varies from 0.86 to 20.8. An average of 76 percent of the 
entire staff are fully qualified journeymen, but this varies with 
facility from 54 to 92 percent. There seems to be no consistent 
relationship between the fraction of the staff needing developmental 
training at these terminal facilities and the training resources. 
The situation is shown graphically in Fig. B-3. 

The history of separations and hires was also examined. These 
are shown in Figs. B-4 and B-5. The pattern of substantial fluc- 
tuation is evident. (It should be remembered that training programs 
are less than a quarter of a year in duration^ and hence quarterly 
data are appropriate.) Again, it is difficult to use these data as 
a basis for future projections. 

D. REMAINING SURVEY RESULTS 

As mentioned, the primary purpose of the survey was to obtain 
the data necessary to help in the development of program costs. The 
data for this purpose are given in Appendix G (starting at p. 218 
where they are used. 
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TABLE INSTRUCTIONAL RESOURCES, HIRES, AND^ ATTRITION 

DURING TRAINING AT 21 MAJOR TERMINALS 



p<?r'Oonta(jo of 
Oporatiorul Cont.TOl In:; 



Cn Rotito (cnior 


FuUy Oual 


New York (CIFRK) 




Cliicaqo 


I. A 
J^i 


Atlanta 


\>{\ 


Miami 


HI 


OaHuG/Pl. Worth 


r •/ 

0/ 


Loi; Aiu;o.lo:; 




Washington Natioiul 


cu 


Detroit: 


/L> 


Boston 


f»9 


Houston 


H.^ 


San Antonio 


y2 


Tainf>a 


GB 


Dcnvoi* 


B9 


Mcmph Is 


80 


Pittsbunjh 


GG 


St. I.onis 


72 


(*lcvolan(l 


7I> 


Minneapolis 


7C 


Kansas City 


04 


Columbus 


92 


Indianapolis 


70 



'^Source: IDA Survey. 
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17. 0 


'j 


J G 
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17.4 
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J 




20 


2 


\7 


9.:^ 


0 


4 


J4 
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0 


3 


1)0 


20. B 


J3 


1 


4 


4./ 


G 


/ 


G4 


JS.O 


•j 


G 


30 


O.BG 


2 


0 


0 


4.3 


0 


0 


0 


9.0 


G 


4 


20 


i.4 


0 


8 


.100 


B.3 


1> 


0 


4S 
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J3 


1 
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JG.i 
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4 


n 


4.7 


G 


y 


64 


7.3 


G 


G 


4G 


B.n 


1 


1 


13 
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3 


0 


0 


9.4 




3 


2/ 
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PERCENTAGE OF OPERATIONAL CONTROLLERS THAT ARE FULLY QUALIFIED 

FIGURE B-3. Relation Between Instructional Resources at Selected Terminals 
and Fraction of Controllers that are Fully Qualified 
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FIGURE B-4. History of ATC Hirings 
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I. INTRODUCTION 



This Appendix is concerned with the capabilities and limitations 
of simulation devices relevant to air traffic control training. For 
background, the history of air traffic control simulation is reviewed, 
and fidelity of simulation--a concept that is central to the use of 
simulators for training--is discussed. Then, the functions of 
National Airspace System, Stage A (NAS-A), Automated Radar Terminal 
System (ARTS) III, and ARTS II are briefly described, and the sim- 
ulcition capabilities and limitations of those systems are discussed. 
There follows a discussion of the need for dedicated simulators at 
the FAA Academy and an examination of future simulation requirements. 
Finally, a summary and conclusions are presented. 

II. HISTORY OF AIR TRAFFIC CONTROL SIMULATION 

The term "simulation" has a variety of meanings, all centering 
around the idea of representing something--an object, a condition, a 
process —in some manner. What is simulated can vary from the simplest 
device to complex systems involving sophisticated electronic com- 
ponents, communication netv;orks, and human decision makers. The 
purpose of the simulation, and thus the method used, can vary as 
well. When the purpose is system design, the simulation may be com- 
pletely mathematical, often utilizing a computer, or it may involve 
an electromechanical mockup. The literature on mathematical sim- 
ulation and on engineering simulation is extensive (e.g., Naylor et 
al.y 1966, and Flagle et al,, 1960) but is not particularly relevant 
when the purpose of simulation is training, performance assessment, 
or development of standard operational procedures and doctrine for 
existing systems. There is also extensive literature in this area, 
perhaps the most comprehensive recent source being Parsons (1972). 
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The idea of using simulation for training is an old one; tho 
game of chess is said to have evolved from ancient oriental war 
games, and training with mockups of military equipment (e.g., wooden 
swords) is centuries old.. The use of complex equipment for simu- 
lation has occurred principally since World War II, the aircraft 
simulator being the best -known example. 

Simulation of air traffic for training has a long history, for 
both military and air traffic control (ATC) purposes. Vickers (1959) 
mentions an ATC simulation study in Australia in 1948. Parsons 
(1972), in his study of man-machine experimentation, discusses more 
than 40 major applications of the technique, many of them for train- 
ing. Some of the more relevant studies are discussed in the follow- 
ing paragraphs. 

« 

Project Cadillac was conducted for the Navy by New York Univer- 
sity from 1948 to 19!?5 using several radar consoles and associated 
communication networks with thirty 15-AM-l target generators. The 
project investigated aircraft surveillance, display, and communication 
problems experienced by operators in Navy Airborne Combat Information 
Centers . 

The Navy in the late 1950s set up a center to be used to train 
personnel for the Navy Tactical Data System (NTDS), a system used 
for aircraft surveillance and intercept direction aboard ship. The 
NTDS computer could generate simulated targets and record system 
performance. Larger, multioperator , computer-based simulation facil- 
ities for NTDS training were established at the Fleet Anti-Air War- 
fare Training Centers in San Diego and Norfolk. 

In a.952-54, the Rand Corporation's System Research Laboratory 
embarked on a study of the behavior of operators in complex systems 
of the Air Force. An early by-product was the realization that lab- 
oratory subjects improved rapidly in performing their assigned tasks 
in a simulated air defense environment. The radar simulation was 
extremely simple— multifold paper was passed in front of a light 
source which revealed printed digits that represented positions of 
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blips on a particular radar sweep. After a series of experiments, 
first with students and then with military crews, a field training 
program called the System Training Program (STP) was developed and 
installed in all Air Force radar stations and in the radar stations 
of the North Atlantic Treaty Organization (NATO) countries. The STP, 
as implemented in the field, utilized a film device for feeding the 
plan position indicator (PPI) scopes a simulated air environment, 
including permanent echoes and weather, and used 15-J-l-C target 
generators controlled by pseudo-pilots. The program was based on a 
combination of well-known and, in theory, at least, generally 
accepted principles, among which were: training a team as a whole, 
stressing the system (in this case, in a simulated mode), and pro- 
viding the participants with knowledge of the results (in this case, 
through an immediate discussion-Lype debriefing after each exercise.) 

During the 1950s, the Semi -Automatic Ground Environment (SAGE) 
system was developed and installed as the primary air defense system 
for the United States. It had a more than superficial similarity to 
NAS-A and deserves some attention, particularly with respect to sim- 
ulation and training. SAGE consisted of 20 or so direction centers 
that were fed digitized radar data from long-range radars which were 
processed by the SAGE computer and displayed on scopes. This man- 
machine system carried out surveillance, identif ica^tion, and inter- 
cept control functions. The system training program that originally 
focused on the radar site was adapted to SAGE and eventually included 
coordinated exercises that involved all radars, SAGE direction 
centers, early-warning aircraft, and appropriate higher headquarters. 
The basic SAGE STP simulation consisted of two parts: (1) pseudo- 
pilots in a special room controlling computer-generated targets that 
appeared on the consoles in the direction center, and (2) a training- 
problem magnetic tape containing simulated inputs from the long-range 
radars as well as. weather information from adjacent sectors and civil 
flight plans from the Federal Aviation Administration (FAA). The 
performance of the system during an exercise was recorded, the data^ 
were reduced by the computer, and additional observations Were made 
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by monitors in a training operations report. Numerous studies were 
made of how to improve the quality of the simulation and, more im- 
portantly, of the training program. While the general approach of 
stressing the overall system through varying the complexity of the 
air defense scenarios proved beneficial, it was also necessary to 
develop ways to stress particular subsystems. For example, to gain 
tighter control over the inputs to the weapons direction section, 
special problems were designed in which the output of the surveil- 
lance section was simulated instead of the radar inputs. This made 
it possible to present the weapons section with situations that it 
would otherwise face only when the surveillance section made mas- 
takes . 

Most of the simulation of air traffic control has, of course, 
' been done by or for the FAA, but a number of studies have been 

oriented toward military problems. One series of studies was car- 
ried out at Ohio State University in the late 1950s under Air Fo.rce ^ 
sponsorship. An electronic simulator was built to display up to 30 * 
controllable targets incorporating altitude effects on speed, wind 
effects on speed and headings, and various aircraft identification 
arrangements. The experiments dealt primarily with human engineering 
and system design considerations and did not deal directly with 
training. 

The work on simulation, first at the Technical Development 
Center (TDC) in Indianapolis and then at the National Aviation Facil- 
ities Experimental Center (NAFEC) in Atlantic City, has been exten- 
sive and is. presumably well known within FAA. Consequently, only a 
few O'f the most relevant studies will be mentioned here. 

The first real-time simulation activities at Indianapolis used 
a simple device that mechanically integrated speed and heading to 
produce a realistic approach path in the form of a small spot of 
^ light which traveled across the surface of a simulated radar scope. 
This device, which simulated a single aircraft, was followed by the 
"Navascreen" in 1950, which projected six controllable targets on a 
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large translucont screen. Each target was operated by a person act- 
ing as pilot. Navascreen was superseded in 1952 by an Air Force 
experimental device known as Teleran, a system using a television 
camera and eight projectors. The projectors were equipped to in- 
tegrate speed and heading to produce realistic curved path^ron a 
large viewing screen. Using a television camera and a specially 
designed flying-spot scanner, the images on the screen were presented 
on scopes in plan position with a rotating sweep. A number of im- 
provements were made over several years. For example, in the TDC 
simulation, 1.5-deg turns in addition to 3-deg turns were simulated, 
and wind drift was introduced. The final configuration had 42 con- 
trollable targets. Most of the work with the simulator had to do 
with terminal ATC problems — developing measure^ of system performance 
and studying problems occasioned by the introduction of new equipment. 

From 1950 to 1958, approximately 50 technical reports were 
prepared fxxm the simulation work at Indianapolis, but apparently 
none of this activity dealt with the use of simulation for training 
controllers (Vickers, 1959). 

The simulation equipment used at TDC was transferred to NAFEC 
in Atlantic City and used there until 1962. In 1960, a new sim- 
ulator. Model A, was acquired from Aircraft Armaments, Inc., and in 
1961, a second one. Model B, was acquired from the same company. 
Model A could simulate three radars; Model B, four. Model A had 48 
pilot positions; Model B, 60. These simulators and their digital 
successors were used in numerous studies investigating such problems 
as simultaneous dual approaches, combining of approach control facil- 
ities, traffic flow patterns, and airportv site selection. Some of 
the later work on investigating system performance measurement is 
discussed in Appendix E. As was the case at TDC, however, no experi- 
mental work was done on simulation for training. 

A very significant development began at NAFEC in 1970, partly as 
a consequence of the Corson Committee ^s strong recommendation that 
adequate simulation be used for training (Vickers, 1972). In a NAFEC 
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program, groups of about 18 students, accompanied by their instruc- 
tors (all from the same en route center), took over the NAFEC Model 
B simulator for assigned three-week training periods. The displays 
were arranged to duplicate two en route sectors from the home facil- 
ity. The traffic samples which were used during this training were 
based on actual recorded traffic for the two sectors. During the 
final week, the traffic density was increased to 110 percent of the 
recorded peak-day density. 

The main object of the customized training was to carry a group 
of students as far along as possible toward checkout as radar con- . 
trollers on the two sectors being simulated. The training also 
stressed the development of the team concept in sector operation. 

The NAFEC program was discontinued in 1972. 

The following comments on the program are quoted from Viekers 
(1972): 

The results have been rather spectacular; but be- 
cause no formal statistical study has yet been completed, 
the FAA Technical Training Division is reluctant to give 
out any figures comparing the progress of students who 
get this customized training with those who do not. How- 
ever, unofficial reports from several sources indicate 
that this threa-week simulation course shortens the amount 
of on-the-job training required for radar checkout by 
about six months. In one case, a group of students from 
the Oakland Center was able to check out as radar control- 
lers, on the two sectors simulated at NAFEC, immediately 
after their return to Oakland, without further on-the-job 
training. 

That such an approach would be successful should come as no 
surprise to those familiar with technical training, but it is un- 
fortunate that statistical data are not available to support this 
finding. It is also unfortunate that it was not possible to con- 
tinue the effort longer, even though it is recognized that the 
Model B simulator was expensive to operate and that NAFEC had other 
missions than to provide training. 
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III. FIDELITY OF SIMULATION 



A major consideration in the use of simulation for training is 
called "transfer of training." That is, how well a person performs 
on the job after being trained on a simulator. "How well" is often 
measured in terms of quality of performance or amount of training 
time and cost required to reach some level of performance 5 generally 
by comparing those trained on a simulator wilh those trained on oper- 
ational equipment. In the training of pilots this issue has been 
under study since 1929, studies becoming increasingly frequent after 
World War II. 

In 1949, a study (Williams and Flexman, 1949) showed that using 
simulators cut training cost by 50 percent and flight hours by 62 
percent. Other studies in the 1950s (Payne et al., 1954, and 
Creelman, 1955) demonstrated that students trained in simulators wore 
as proficient on flight checks as students trained in aircraft? made 
74 percent fewer errors, and showed superior ability in approach and 
landing. In the 1960s , numerous studies addressed the effectiveness 
of different degrees of completeness of the simulation and the roles 
in flight training to be played by various part-task training de- 
vices and operational flight simulators. As the name implies, a 
part-task trainer is limited to some part of an entire task to be 
learned (e.g., trainers for pilot navigation or radio procedures 
rather than a complete cockpit simulation). Questions about what 
training could be done most effectively in cockpit procedures trainers 
and in instrument and contact flight simulators, rather than in com- 
plete operational flight simulators, were raised and answered. In 
the interest of safety and economy, the amount of training in flight 
was minimized. 

Between 1966 and 1971, American Airlines reduced flying time 
for Boeing 707 captain transition training from 18.3 to 3.1 hours 
(Moran, 1971).. The comparable reduction for the Boeing 727 was from 
20.6 to 3.5 hours. In 1971, captain transition training for the 
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Boeing 747 averaged 5*3 aircraft flying hours and 22.1 flight sim- 
ulator hours; such training Cor the DC-10 averaged 2.1 aircraft 
flying hours and 19.5 flight simulator hours. The other major air- 
lines have had similar success,- and the FAA Flight Standards Service 
now permits almost all transition training for the airlines to be 
accomplished in simulators. 

The central importance of simulation in flight training is well 
accepted. Considering the fact that air traffic control radar sim- 
ulation is technically muchljimpler, it seems strange that it has not 
played a similar role in the training of FAA controllers. Each of 
the three militaiy services uses simulation extensively in controller 
training, and, as has been mentioned, the technique is an integral 
part of air defense training. 

While simulation is the process of "representing" something, it 
is clearly implied that a simulation need not be a complete duplicate 
of the real thing. The degree of simulation, what portions of a 
system are included, and the fidelity of simulation will vary with 
purpose. A key element in using simulation for training is the 
nature of the task or tasks to be learned. Training to impart pre- 
cise motor skills gives rise to requirements different fran those of 
teaching general procedures. In the former case, it might be critical 
that the sensory cues in the simulation be very realistic, and it 
might not be so important to include parts of the system not directly 
associated with the sensory motor activity of the operation. Another 
consideration is the stage of training for which a device will be 
used; early stages of learning are often best done in a simplified 
environment, the complexity to be found in real operations being in- 
troduced later. Just what features of the real world should be in- 
cluded sometimes can be answered only by research. A question such 
as what degrees of freedom of motion should be built into a flight 
simulator does not lend itself to answer by analysis alone. 

Prophet (1966) compared a procedures trainer that cost $100,000 
with a plywood mockup of a cockpit with a photographic instrument 
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panel that cost $100. The static mockup was as effective as the 
expensive trainer in teaching cockpit procedures. It was also as 
effective on tasks such as reading instruments and making precise 
contrfDl settings. 

Cox et al. (1965) investigated transfer of training in a study 
that involved 12 different devices, each of which simulated a com- 
plex control console in a missile system. The most realistic device, 
the "hot panel," was the same size and shape as the real console, 
with every light, switch, meter, intercom, and telephone functioning, 
A "cold panel" was the same, except that no electrical power was 
used. Other variations were a panel made of cardboard, a full-gize 
black-and-white photo of the face of the console, a line drawing of 
the panel, and panels differing in size. The results were that for 
training in fixed-procedure tasks, the fidelity of the simulation 
did not matter. Grimsley (1969) later verified these findings by using 
three versions of the console — high, medium and low fidelity. This 
time the emphasis was on retention and retraining time. There were 
no differences in initial training time, in retention after four and 
six weeks, or in time to retrain to the criterion. 

An interesting study v;as done rec^ently (Koonce, 1974) in which 
statistically significant results were obtained indicating that 
better simulation does not necessarily mean better transfer of train- 
ing. The experiment involved testing a pilot's instrument flight 
skills on successive days. Training was in simulators on the first 
two days and in an aircraft on the third day. There were three groups 
of pilots. For one group the motion system in the flight simulator 
was turned off; the other two groups wei'C trained under two different 
levels of sophistication with respect to motion cues. There was the 
usual finding that motion cues, simple or complex, make the simulator 
easier to fly. But when the groups were tested in the aircraft, the 
group that had experienced less realism — no motion in the simulator — 
was significantly better. Apparently the motion cues in the simu- 
lator were misleading, even though the motion system had apparent or 
"face" validity; that is, pilots thought it realistic. 
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Chapanis (1972), in commenting on the dangers of assuming valid- 
ity of a simulation, gave a striking illustration of how "realistic" 
simulations sometimes have unintended features. Many highly realistic 
driving simulators induce motion sickness even in people who never 
experience motion sickness in driving real automobiles. The curious 
thing is that all attempts to isolate and remove the cause or causes 
of this have been largely unsuccessful. 

The results obtained from simulation have to 
be interpreted and extrapolated to real world 
situations with great caution. In the final 
analysis, the validity of a simulation has to 
be proven experimentally. It cannot be taken 
for granted, no matter how impressive, inter- 
nally consistent or elegant the simulation. 
(Chapanis, 1972, p.- 726) 

Thus, realism does not necessarily mean good training. What is 
important is that the right aspects of the real environment be present 
for the task to be learned. In the case of the radar portion of con- 
troller training, the realism of the display itself is probably not 
as important as having all functional aspects of the system repre- 
sented, so that communication procedures and other basic skills can 
be practiced in safety. It is probably obvious that if the sim- 
ulation is too poor, motivational problems will arise and the benefit 
that comes from a student ^s becoming thoroughly involved, with a 
problem will be lost. 

An important consideration is stimulus control and the ability 
of a trainer to create situations in which the proper behavior can 
be taught. This makes it possible to present situations that occur 
infrequently in real life and to give practice unobtainable in a 
reasonable period of time on the job. It also makes it possible for 
the simulation experience to be ordered coherently in terms of the 
materials to be learned. 

It seems clear that using a simulator in early phases of train- 
ing would be effective even if it is lacking in realism and speci- 

i 

ficity. Military experience with complex command -and -control systems 
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certainly suggests that working with the actual system, but in a 
simulated mode, is an effective final stage of training and is the 
indicated way to maintain or to increase proficiency after becoming 
qualified. In the case of both NAS-A and ARTS III, utilizing the 
built-in simulation capability as a final stage of radar training 
^^xL obviates the problem of keeping the bimulation current, a problem 
faced by separate simulators as new features, particularly features 
of a decision-aiding nature, are added to e system. This will be 
important. There are problems with utilizing operational equipment 
for training, however. These problems will be discussed in later 
sections of this Appendix, 

IV. FUNCTIONAL SYSTEM DESCRIPTIONS 

A. NATIONAL AIRSPACE SYSTEM, STAGE A (NAS-A) 

NAS-A uses air route surveillance radars (ARSRs) (200-mile) and 
their associated air traffic control (ATC) radar beacon interrogators 
(RBIs) in a CONUS network in which the wideband (video) data is 
digitized at the sensor and transmitted over voice-quality circuits 
to central computer complexes at 20 air route traffic control centers 
(ARTCCs), Aircraft tracks are computed for all aircraft and sent to 
the display console (Fig. C-1), 

The system includes flight plan entry, progress reports, flight 
plan updating and forwarding, plan position and tabular displays, 
and semiautomatic transfer of control between facilities, 

NAS-A technically represents the highest level of ATC automation 
by reason of the radar level and beacon level tracking, the multi- 
sensor and multiprocessor capability, and the automation of flight 
plan functions. The Automated Radar Tern[iinal System (ARTS) III is 
less automated, and the ARTS II is even less automated. 
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B. AUTOMATED RADAR TERMINAL SYSTEM III (ARTS III) 

The basic ARTS III (Fig. C-2) is based upon system work initially 
started at the Atlanta terminal and now programmed for full operation 
at 61 terminals. The terminal radar approach control (TRACON) sycLem 
functions at instrument flight rules (IFR) rooms with tower cab 
(TRACAB) displays where appropriate. 

The block diagram shows that beacon-equipped aircraft are auto- 
mated to beacon tracking level, while non-beacon-equipped aircraft 
are not tracked. When the radar video is digitized, all aircraft 
can be tracked, as in NAS-A. 

C. AUTOMATED RADAR TERMINAL SYSTEM II (ARTS II) 

For less busy terminals, the level of automation and the size of 
the facility are constrained by basic economics to be somewhat less 
than those of ARTS III. See the simplified block diagram of ARTS II 
(Fig. C-3). ARTS II employs beacon data level (not tracking level) 
sensor inputs to a minicomputer which, in basic form, is expected to 
handle up to six displays. 

ARTS II is in procurement, and to date there has been no pro- 
gram development for training purposes such as the enhanced target gen- 
erator (ETG) of ARTS III. While ARTS II is much smaller and cheaper 
than ARTS III, it could very likely support training at most, if not 
all, of the locations where it is installed. The low traffic count, 
the small number of ATC displays, and the modular expansion capability 
of the minicomputer support this view. 

D. ENHANCEMENT PROGRAMS 

From a simulation viewpoint, there are three levels of sensor 
input: beacon data level (in ARTS II), beacon tracking level (in 
ARTS III), and both beacon and primary radar tracking ^evel (in NAS- 
' A). Plans and programs to provide tracking level on all aircraft 
(as in NAS-A) will materialize as program priorities, technical capa- 
bilities, and operational capacity will allow. 

107 

ERIC 




108 
1.11 



MAP 
VIDEO 



PRIMARY 
RADAR 



PLAN 
POSITION 
INDICATOR 



WIDE BAND 



SECONDARY 
RADAR 



IO-??-74-l9 



BEACON DATA 
ACQUISITION 
SUBSYSTEM 
(B DAS) 




BEACON 
CONTROL 
UNIT 



BEACON DATA LEVEL 



MINICOMPUTER 



AM KEYBOARD 

^ 

PEM 



FIGURE C-3. Simplified Block Diagram, ARTS II 



V 

FOR 
S 

GENE 


N 

MAT 
^ATOR 






BRITE CONTROL 
& 

A/N CONTROL 
PANELS 



109 



112 



As the system evolves into the "upgraded third-generation" 
phase, the surplus capacity at many locations is expected to gradually 
diminish until competition for main frame time militates against use 
for training and requires reconfiguration of the system. 

Projected operational improvements which may be very demanding 
on processing capability are exemplified as: 

1. ARTCC 

a. Conflict detection 

b. Conflict resolution 

c. Intermittent positive control (IPC) 

d. Discrete address (tactical) data link of discrete address 
beacon system (DABS) 

e. DABS interrogation hierarchy control. 

2. Terminal Radar Approach' Control (TRACON) 

a. Automatic radar level tracking with tagged targets 

b. Metering .and spacing 

c. Conflict prediction 

d. Conflict resolution 

e. Fail safe/Fail soft/Auto standby switch 

f . Multiradar 

g. Multiprocessing 

h. DABS data link. 

3. Tower Radar Automation Cab (TRACAB) 

a. Radar level tracking 

b. Noise abatement pattern control 

c. Wake turbulence separation control 

d. Multisegmented approach and departure routings to exploit 
the characteristics of area navigation (RNAV) and micro- 
wave landing system (MLS) 

e. Upgrading ARTS II from beacon data level to beacon tracking 
level. 
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Thus it VADuld appear desirable to view the training situation 
^within the context of both short- and long-term needs. The short- 
term needs arc discussed in Section V, where the simulation 
capabilities of NAS-A, ARTS III, and ARTS II are reviewed, and in 
Section VI, where dedicated simulators are considered. The longer 
term situation is discussed in Section VII, where a systems approach 
to simulation is advocated to cope with training requirements in the 
evolving National Airspace System. 

V. SIMULATION CAPABILITY OF NAS-A, ARTS III, AND ARTS II 

NAS-A and ARTS III were initially delivered with a simu3ation 
capability designed to permit equipment checkout, to facilitate 
maintenance, and to be used for initial facility shakedown. This 
simulation capability, however^ did not permit operation of the 
equipment in training and operational modes at the same time because 
the simulated targets appeared simultaneously on all operating scopes 
in the facility. Because of these limitations, staffs of two facil- 
ities (Houston ARTS III terminal and Washington NAS-A center) were 
authorized and tasked to develop "patches" to the programs that would 
permit flexible training at designated positions without interfering 
with operational positions. The ARTS III patch, known as the 
Enhanced Target Generator (ETG), has been approved and is now being 
implemented; the NAS-A patch to the Dynamic Simulation (DS) program-*' 
is developed but has not as yet been approved for implementation at 
all centers. 

With the Enhanced Target Generator (ETG ARTS III) and the Dynamic 
Simulator (DS NAS-A), designated positions called "pilot" positions 
can be used to generate simulated targets which can be displayed at 
other designated positions independent of all other operational 



Not yet completed for computer display channel (CDC) version of 
NAS-A as of October 1974. 
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positions being used for control of r^al traffic. The simulated 
targets will not appear on any of the operational scopes, and the 
simulation will not interfere in any way with the use of the oper- 
ational positions. The simulation programs (ETG and DS) utilize part 
of the equipment capability of the installed ARTS III and NAS-A, 
however, and in certain circumstances reduce the capacity of these 
systems to handle operational traffic. For example, should the 
operational traffic load in NAS-A increase to the point where it 
equalled the number of tracks for which a particular center was adapted, 
the system would gradually reduce the number of simulated tracks 
available, but there would still be an impact on processing times. 

The simulations can provide realistic representations of real 
traffic for digital operations but not of analog radar. All "static" 
data that are available in the facility computer memories as well as 
live operational traffic can bo presented at the training positions. 
The "static" data includes: sector boundaries, airport positions, 
navigation aid positions, holding patterns, airways, and weather 
contours (NAS-A only), all of which can be displayed optionally by 
operator selection at any of the simulation positions. Essentially 
all of the operations required in operational control of aircraft 
can be simulated, and outputs can also be generated for assistant con- 
troller and support positions, as needed. As part of a simulation 
problem, tapes of scheduled simulated traffic can be prepared and 
entered into the computer before the problem begins, and flight 
strips will be printed out at the appropriate simulated support 
position. Also, simulated traffic entered into the simulated areas 
by the "pilots" will be associated with the stored traffic data, and 
tracks will be started automatically on targets that pass the system 
association criteria. All positions will be connected by the stand- 
ard level 300 communications system, and "pilots" will control sim- 
ulated targets in accordance with the trainee controller's instruc- 
tions; thereby, the pilot -controller communications are also sim- 
ulated . 
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Apart fran the analog radar presentation, the principal limi- 
tations of the simulations are: (1) availability of positions and 
capacity in an operational facility; (2) the ability of "pilots" 
to generate and control adequate numbers of simulated targets; (i) 
the inability to simu].ate adequately the interactions with adjacent 
facilities; and (4) the lack of a recording capability designed for 
training purposes. In addition, the simulations have limited flexi- 
bility in that they can be interrupted but ^cannot be "backed up" or 
have parts repeated in the middle of a simulation problem to point 
out a control error or a deviation from control rules or standard 
practices . 

The present simulations do not include a capability to display 
broadband radar data. In NAS-A, broadband simulation could be 
obtained by suppressing all alphanumeric and other symbol ogy, other 
than a symbol representing each target position, e.g., a slash (/). 
For ARTS III, for which the normal mode of operation is mixed digital 
and analog data, the position of each controlled target could be 
represented by several symbols: the usual alphabetic symbol for the 
controller position and additional syrabols [e.g., a slash (/)] in 
proximity for the analog radar and beacon return. 

Although complete system fidelity may not be required for 
training, the use of a slash for broadband returns departs consider- 
ably from realism. The true analog return is an arc, concentric with 
the center of the display. Its length will vary with distance from 
the sensor, and it may be broken, depending upon the sensor ^s antenna 
pattern and the suppression of individual beacon replies. In addition, 
deleterious effects such as "ring-around" (i.e., a circle, or broken 
circle, arising from sidelobe effects) and "spiraling" (arising from 
asynchronous interference from neighboring beacon interrogators) may 
occur under actual operational conditions and may degrade the con- 
troller's display. 

Table C-1 summarizes the capabilities of the two simulation 
programs . 
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TABLE C-1. CHARACTERISTICS OF SIMULATION 



Dynamto ntmulntor (»AS-A ) 



run<:t:i<)ns 

'^Smii Virion tnnnin.i»»' (m.iiMMl/-iuU)ifnl Ir! 

.in<l innil.lli;«l I.ImI i i<- 

Hixi^l Uv>! 'iit<l siimil.Ur^l InUflc 
4it «:itiiiit.il I'ti |K>'!!tioin only 

■■ ir«j«:l iMiliy/t enti I nation 

'llMC'k ill! I {•tttf 

(^mpiH M- >ir.^ori.ition--l !'-ii*'"lii \ \ I 
If ili«l-ol f 

rUijIlt «'iiti>f/"«>'IU mmUoii 

iiii.i MiM-k: run 

ffoni: 

01 hItM I 

rjiM-n|«'ii' y/i^lii1o l.i i Inn 

\' i«l.ir WhIi"> 

AlviliXt IVI^I«ll*/I^^V»' 

hiijil-il r.Ml.ir/lM'.inin 
l>-aiUi'/l«M» «>ii (Itl ijt/"i''ii»> 

'IVlMjrl iiniiiiMM' 

Wvh' './A 

l>»r.irioil (pl-ni) 
All i(n<l(* 

Atlitii^lo, i.iiii«m'Ui<li><l 

Picnt ity 

'^pec'l 

A«;rnler«il idit 
ikM'iini) 

'hii-i> r.iH»/'i<'W lio.nl Inij 
Noiso: pi in (o^Uion 
Win<laijfi 



Yon 
Yoj; 

Vo"., "'pilot "/* r.iinno ix)siii()n^ 

Yiv. 
Yi-n 
Ml I 

/ 
/ 
./ 
/ 



llo 
IIHI 

Alp!Lilii>l if f.yiilini 
WAi^ (' (PDIO <uily 
Y<!S, iiioln^Mmi syifilM>l iu ri»H (M> 
Yor. (rniO 

MiMifis t*, './A (nm/iim> 

ViM'lor line 
('<lrp < Uin^ji' to iiixl. *.p<'«*<l) 

Yi'S, ^jp^o'l «Iepon*l0Ml 

m 
no 

Y<»'S 



• Yor. 

Yos (no I iv- M-mI-.) 
Y"S 

Y<!«;» "pi lol."Ar.iin<-^ jojiitious 
Y«'S ^xjn only) 
YO". 

^(•r. (n" ( UmIii pi'"!'!) 

./ 



Yos/Yor. 
II<»/H<» 

J 
J 
J 

Aiplulii'l i' .viril»«>l 
WhIo O (PIUO only 

Y<!1 

Yn.r. (nor in 

n^U's 0, './A (i'»)ii/i-nn> 

rnit (mly 
Y<-^ 
Y<»s 

Yes, 5pce«l <lcpc»''«ni 
Yfs 
Y'»s 



**{.in Aniiidlo l^ll'•|<«l■; jml HvjcKi- ( 1 1 ii|lil pKin:.. 

'^:;imiilijl inn ovrrt iHr im«h"— twy h<- inHinliil lof wtili -Uind j.imMlJlitHi. 



114 



ERIC 



117 



I 



Using DS and ETG for training clearly requires special efforts 
to minimize any negative transfer caused by the lack of realism of 
the broadband display. Since the developmental has ample opportunity 
to observe real displays, however, this should not be a groat problem 

The evolution of the ATC system and the introduction of the 
enhancement programs listed in Section IV-D of this Appendix may 
overload the automated systems and change training capabilities. An 
analysis of the present air traffic volume with respect to current 
capacities (FM, 1974a) does not indicate any system capacity problem 
with using DS and ETG for training in facilities. In Table C-2 some 
of the pertinent characteristics of the NAS-A system at centers are 
tabulated. As an example of the distribution of operations during 
the day, Los Angeles is cited and shown in Figure C-4. The data 
shown are for the peak day of 1973 (traffic was less on all other 
days). There may be scattered occasions when training would be inter 
rupted, but in general, with the possible exception of the case wher^ 
all training is' done at facilities, it appears that training could be 
done on a .scheduled basis during normal working hours. 

If all radar training were done at facilities, ope??ational 
changes, such as operating the automated system longer each day, 
would probably be needed. 

The ARTS II system, based upon the prototype tested at Wilkes- 
Barre/Scranton Airport, uses the same beacon data acquisition system 
(DAS) as ARTS III and, like ARTS III, does not process the primary, 
radar video data from airport surveillance radar (ASR). 

DAS contains circuitry which can generate aircraft targets 
fixed in range and bearing for maintenance troubleshooting. Two 
modes are supplied: 

1. A single aircraft generated at each pulse repetition 
frequency with selectable range decodes. 

2. Multiple processing of 16 equally spaced aircraft in 360 
deg of azimuth but all at the same range. 
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Provisions are made on the maintenance panel for" generating sim- 
ulated azimuth data pulses and mixed aircraft pulses in the main- 
tenance mode. Either beacon Mode 3/A (identification) or C (altitude) 
is selectable. 

TABLE C-2. TRAFFIC (PEAK HOUR OF PEAK DAY, 1973) AND 
TRACK CAPACITY AT CENTERS 







9020 


1973 


PEAK 












VERSiaN 


IFR 


HOUR 


TRACK 






AUTHORIZED 




& DISPLAY 


(millions) 


OPERATIONS 


CAPACITY 


SECTORS 


PVD's 


CONTROLLERS 


ALBUQUERQUE 


A 


CDC 


0.91 


275 


300 


29 


46 


280 


ATLANTA 


D 


CDC 


1.44 


365 


400 


46 


54 


469 


BOSTON 


A 


CDC 


U.V/ 






91 

L 1 


JO 




CHICAGO 


D/E 


DCC 


1.65 


410 


500 


38 


69 


566 


CLEVELAND 


D/E 


DCC 


1.73 


535 


400 


43 


65 


573 c 


DENVER 


A 


CDC 


0.64 


200 


300 


30 


44 


273 


FT. WORTH 


D/E 


DCC 


1.23 


430 


350 


39 


57 


338 , 


HOUSTON 


A 


CDC 


1.15 


600 


300 


38 


60 


424 


INDIANAPOLIS 


D 


CDC 


1.29 


340 


400 


*33 


49 


450 


JACKSONVILLE 


A 


CDC 


1.08 


375 


400 


35 


57 


407 , 


KANSAS CITY 


D 


CDC 


0.99 


335 


400 


36 


52 


365 I 


LOS ANCaES 


D 


CDC 


1.05 . 


325 


250 


34 


48 


361 


MEMPHIS 


A 


CDC 


1.09 


394 


250 


30 


42 


348 - 


MIAMI 


A 


CDC 


1.05 


365 


350 


29 


50 


293 


MINNEAPaiS 


A 


CDC 


0.91 


264 


250 


28 


41 


316 


NEW YORK CITY 


D/E 


DCC 


L61 


421 


450 


37 


62 


576 


OAKLAND 


A 


CDC 


0.91 


278 


250 


29 


44 


346 


SALT LAKE CITY 


A 


CDC 


0.41 


105 


250 


18- 


43 


183 


SEATTLE 


A 


CDC 


0.60 


163 


250 


16 


35 


185 


WASHINGTON. D.C. 


D/E 


DCC 


1.37 


400 


400 


34 


63 


482 


TOTAL 






22.07 






649 


1039 


7650 


AVERAGE 






1.10 






32.5 


56.0 


382.5 



Sources: FAA Sizing Committee. FAA Air Traffic Activity. IDA Survey. FAA AAT-110. 
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FIGURE C-4. Los Angeles Air Traffic on Peak Day, 1973 
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Simulation for training has not been a requirement in the ARTS 
II prototype. Production systems are still in the early stages of 
procurement. 

The minicomputer of the production ARTS II is of modular form, 
and it is unlikely that there would be an inherent system limitation 
to providing an enhanced target generator (ETG) similar to that of 
ARTS III. In its basic form, ARTS II is required to handle 2b6 air- 
craft and six display consoles, and it can operate utilizing data 
from (a) local ASR site, (b) remotely sited ARTS III (from DAS through 
Modems), or (c) NAS-A production common digiti:ier (PCD, beacon por- 
tion). 

The position entry module (PEM) can be employed to move a single • 
target manually across the display. 

The development of programs to provide simulation equivalent to 
ARTS III ETG could start immediately, since the processor selected 
for ARTS II is an off-the-shelf commercial item. 

VI. DEDICATED SIMULATORS 

An alternative to using the developed simulations for NAS-A und 
ARTS III is a separate simulator dedicated to training. The require- 
ment for such simulators goes beyond the simulation capabilities of the 
ATC systems themselves. In large part, it depends on what portion 
of radar training is to be done in the field. If only sector qual- 
ification and remedial and refresher training are done, dedicated 
simulators do not appear needed, because of the simulation capa- \ 
bilities of the present automated equipment and because of the prob- 
lem (discussed later) of keeping separate simulators current. If 
all radar training is done in the fieJd, the training load may ex- 
ceed the system capacity available for simulation in the near term, 
particularly at terminal facilities. In fact, the alternative of 
full-time training at a terminal (as offered in Appendix H) is not 
really feasible for the large number of IFR terminals with only 
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two or three people, and with few, if any, instructors. For this 
reason and for standardization and other considerations, some degree 
of centralization of radar training is^ indicated. To do this, the 
FAA Academy (or regional training centers) must have a radar sim- 
ulation capability. 

The Servonics simulation equipment still in use at the Academy 
and a few other locations is quite inadequate. Its replacement could 
probably be justified in terms of annual maintenance costs alone, but 
in addition it has inadequate capacity and availability because of its 
low reliability and the scarcity of replacement parts. 

One option for replacing the Servonics equipment can be quickly 
dismissed on the basis of cost, and that is installing the necessary 
additional hardware to be able to operate NAS-A and/or ARTS III in 
Oklahoma City in a simulated mode. For NAS-A hardware, this cost 
would be on the order of $4 million to $5 million." The Academy would 
still lack an effective broadband simulation capability and would 
suffer from most of the limitations of the current dynamic simulation 
program. In addition to thv^ hardware costs, there would be some 
unknown costs for software. Another unexplored area is how available 
the Academy's 9020 would be for controller training. 

The situation with respect to using an ARTS III. system at 
Oklahoma City is probably similar, if less expensive. Neither of 
these alternatives would be economical in operation, and both would 
require extensive manpower for pilot's, hand-off, etc., as compared 
to a device or devices designed for training. As discussed later, 
training should get more attention as a design consideration in the 
future evolution of the ATC system. As a part of this, the FAA 
should explore ways of utilizing standard FAA hardware, particularly 
plan view displays (PVDs) and minicomputers that will be in the 



'.^'internal FAA estimate (FAA, 1973c). 
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inventory, and a minimum of special-purpose equipment to achieve the 
simulation capability needed at various locations at various times- 
This would make it possible to meet shifting simulation requirements 
without large sunk costs. 

Such an approach does not meet the immediate need for a radar 
simulation capability at the Academy, however. The number of trainee 
positions that are needed and the specific features (data blocks, for 
example) that should be included in the simulator depend on tho extent 
to which radar training is centralized. As indicated above, most IFR 
terminal training probably should be centralized; whether en route 
training is done all in the field or partly at the Academy depends 
on factors discussed elsewhere in this .report, 

A simulator equipped for terminal training is needed at the Acad- 
emy. Whether it should also be equipped for en route training, whether 
the NAS-A enhancement should be done, and whether it should be expanded 
beyond two-sector capability should be determined as decisions are made 
about further centrali^^ation (see Chapter VII in main report), 

VII. FUTURE SIMULATION REQUIREMENTS 

The FAA apparently did not, in the design of KAS-A and ARTS III, 
recognize training as a function that should be reflected in system 
design. In NAS-A, the dynamic simulation capability was designed 
for use in system checkout, not training. Although training is 
mentioned as a reason for the program, it seems clear that no real 
consideration was given to training requirements. Certainly the 
most cursory analysis would have indicated the necessity of being 
able to isolate simulated traffic from live operations. The ARTS I'll 
situation is similar. In both cases, recognizing training require- 
ments as legitimate design considerations could have resulted in 
much more powerful simulation tools than were actually produced. 
Support for this statement can be drawn from the fact that it was 
possible to patch both programs in such a way as to make them useful 
for training, 
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Furthermore, even though it is now recognized that the enhanced 
target generator and dynamic simulation programs as modified are 
valuable training tools, no effort is being made to improve them in 
future releases. There are modifications, such as more realistic 
speed changes, that could be included at little cost in the rewrite 
of the dynamic simulation program that is presently under way. 

Unfortunately, the same oversight seems to obtain in the cc je 
of ARTS II. A brief analysis of the system specifications again 
indicates an absence of training considerations. The simulation 
capability is even more limited than in ARTS III, although the nature 
of the processor would make it relatively simple to have a sophis- 
ticated simulation program. 

It is important for future system effectiveness that simulation 
for training be included in the design of future versions of the 
National Airspace System. The problem is more complicated than that 
of procuring dedicated simulators or including on-line simulation 
programs . 

From the viewpoint of the national program, it would be an over- 
simplification to assume that dedicated off-line special ATC sim- 
ulators can retain currency and accomplish optimum training results. 
It probably would be equally fallacious to assume that training 
programs can succeed based solely on the allocation of time and hard- 
ware modules within existing operational ATC systems. In the future 
it may be impossible to maintain training schedules at many locations 
when the traffic levels are too high, when the equipment maintenance 
status is too low, or v/hen enhancement programs are behind schedule. 

With the ever-increasing power and flexibility of minicomputers 
and their dramatically decreasing cost per computational function, 
one needs to review very carefully the long-term advantages of com- 
peting for time on, a central computing facility. This is especially 
true when such a facility has an architecture predating that of 
current multiple-access distributed modular systems and, in particular, 
may lack asynchronous bus and bus controller performance capabilities. 
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For reasons of economy and realism, ATC controller training 
should take place on standard FAA consoles out of the inventory 
rather than on special simulator consoles which might be quite dif- 
ferent. The "pilot," instructor, and support positions, however, 
need not be FAA standard operating consoles. Target control and 
monitoring, "pilot" air/ground voice subsystem, and video generation 
for standard FAA displays can be independent modules not normally in 
use in the ATC system. 

Because of planned upgrading and enhancement programs, the sub- 
systems should be modular, so that add-on modules can be utilized to 
upgrade the training as automation and its related new training 
requirements develop. 

The simplified diagram in Fig. C-5 show.s the alternatives 
available in the present ATC system. Broadband, hardware -generated 
video information can be fed at position A for use in the three cat- 
egories of display shown. This information can be controlled via a 
target control and monitoring^ subsystem connected to a video gen- 
eration subsystem, or it can be input from storage via video tape, 
film, or TV camera. Map overlays can utilized at the camera or on 
the final display itsolf . 

At position B the controls are available to simulate live tar- 
gets using software programs of the data processing system, such as 
the enhanced target generator (ARTS III) and dynamic simulation 
(NAS-A). 

Position C identifies the fact that digital data in R DAS, PCD, 
or B DAS digital content and format can be fed from a voice -bandwidth 
source and appear on the displays as simulated aire aft with alpha- 
numeric data blocks . 

It is recommended that ATC hardware implementation schedules 
include extra standard subsystems and items to cover training needs, 
that these items be inventoried with nnt^mal operational equipment, 
and that training subsystems be withdrawn, configured, utilized, 
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and returned to the operational inventory as the training load 
increases and decreases. 

Future simulation requirements to cover many of the projected 
operational improvements in the ATC system cited in Section IV-D of 
this Appendix can be met by utilizing standard subsystems of the ATC 
system with appropriate software packages and target generation and 
control equipments , 

Processing for simulation can take three compatible forms: (1) 
on-line NAS-A and ARTS III, (2) on-line NAS-A and ARTS III supple- 
mented with "smart" consoles containing "naked mini" computer cards 
to unburden the on-line ATC processors, and (3) off-line or dedi- 
cated training simulators utilizing the same "naked mini" computer 
elements packaged as an identifiably separate minicomputer. Use of 
ARTS II processing and interface components would simplify the sub- 
sequent logistic and maintenance problems while maintaining reliable 
simulation. 

•♦ 

ARTS II subsystems are modular, their tloorspace requirements 
are small, and the equipment is readily transportable. With flexible 
cabling and appropriate plugs and receptors:, the hardware could be 
placed at the position selected and operated whenever classroom sim- 
ulation needs develop. This practice, of course, would be imprac- 
tical with the NAS-A or ARTS III processing ccnplexes . 

Requirements for future simulation involving processes such as 
conflict detection, conflict resolution, and intermittent positive 
control could cause the ATC displayed information to be quite dif- 
ferent than present NAS-A and ARTS III. Placing an automatic or semi- 
automatic data link in operation to reduce ibtnd change the functions 
of air/ground voice circuits would also result in display changes. 
However, each enhancement or change usually relegates the older 
processes to a backup role. Hence, the simulator's load increases, 
and the number of backup modes the prospective controller must deal 
with increases. A systems approach to designing the required sim- 
ulation equipment appears to be essential. 
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Thus, there will be a, continuing need to utilize live, taped, 
and controlled target analog video in training simulation. Each step 
in digitizing, tracking, and using the processed data and tracks will 
be required in simulation at some stage of the controller's curriculum. 

As the ATC enhancement programs mature and new features become 
available, single-thread processing will give way to multiprocessing. 
This provides increased reliability to some extent by reserve and 
available reserve capacity with automatic changeover when trouble 
develops. This factor allows one, on a long-term basis, to plan for 
increased on-line simulation training with some confidence that com- 
petition for frame time for operational purposes will not interfere 
with training schedules. 

Changes in the state of the art of information processing have 
caused equipment costs to run counter to prevailing inflationary 
trends for several years. The new large-scale integrated-circuit 
(LSI) systems are quite powerful, yet comparatively low in cost. 
Since the equipment is reliable and has a long life, the overall 
training costs reside primarily on the personnel side. This was not 
true a decade ago. From an overall FAA viewpoint, it is probable 
that program cost-ef f ectiveness can be enhanced through the procure- 
ment and utilization of compatible simulation hardv;are and software 
supported at a much higher priority than in the past. 

A centralized group with knowledge of training needs, available 
ATC operational hardware and software, and competence in minicomputer 
applications could provide the systems approach to applications of 
simulation in training which is needed to cope with future simulation 
requirements . . 
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VIII. SUMMARY AND CONCLUSIONS 

The following statements sununari^.e this Appendix and prcsont its 
conclusions: 

• Analysis of training by moans of simulation in a wide 
variety of fields ir.dicatcs the danger of making general- 
izations that are not based' on careful .research. It does 
appear, however, that for training purposes, completeness 
of a simulation— what elements of a system are included — 
is more important than the fidelity or precise realism of 
any particular element. 

• ATC simulation has been extensive in the FAA, but eval- 
uation of its use for training has not been done, and 
should be. 

• Simulation required for training was given little or no 
apparent consideration in the design of NAS-A and ARTS 
III. The same lack of understanding of the importance 
of an operational simulation capability appears to apply 
in ARTS II. Effort should begin immediately to provide 
an operational simulation capability in ARTS II. 

• The simulation capability that now exists in NAS-A and 
ARTS III, while less than ideal, appears to be adequate 
for training in the near term. Because of the importance 
of broadband in ARTS III, special consideration needs 

to be given to integrating simulation training with OJT 
involving live radar. 

• If the FAA Academy is to play any role in radar training, 
an improved simulation capability there is imperative. 

In the case of IFR^ terminal training, it seems clear 
that some degree of centralization is needed. If this 
K training were done at regional locations instead of 
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the Academy, simulation capability would be required 
at those locations. 

Evolutionary changes in air traffic control and possible 
future traffic loads require that future simulation re- 
quirements' be addressed. It is recommended that this 
be done as an integral part of future system develop- 
ment. A promising approach is to investigate a mod- 
ular system involving standard EAA equipment including 
minicomputers such as those being procured for ARTS II, 
interchangeable software, and a minimum of special- 
purpose hardware. 
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A. INTRODUCTION 

The purpose of this Appendix is to review procedures currently 
in effect for the selection of air traffic controllers. 

Certain characteristics of the individuals accepted for training 
will affect their ability to complete their training, how well they 
perform on the iob, and how long they continue to work as con- 
trollers. Some tradeoffs can be made between the selection standards, 
the method and amount of training, and the qualification standards 
for a journeyman controller. Some of the interactions between 
selection and training will be considered here. Fortunately, the 
Federal Aviation Administration (FAA) has long recognized the impor- 
tance of selection and has conducted many studies in this area. This 
Appendix draws on an outstanding series of reports prepared for the 
FAA by Education and Public Affairs, Inc. (EPA) in 1970-1972 as well 
as many studies done by the Civil Aeronedical Research Institute/ 
Civil Aeromedical Institute (CARI/CAMI) since 1961. 



B- METHOD OF SELECTING CONTROLLERS 

The selection of controllers uses the following procedures and 
criteria : 

1. Civil Servjce Commission (CSC) Air Traffic Control (ATC) 
Aptitude Tests 

These include tests in 

• Computations 

• Special patterns 

• . Following oral directions ^ 

• Abstract reasoning and letter sequences 

• Air traffic problems. 

Duration: about 2.5 hours. 
Minimum acceptable score: 210. 
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2. Rating of Experience and Education 

Various ccsmbinations of education and experience are graded 
as follows: 

GS-4: 2 years undergraduate study or experience 

GS-5: 4 years undergraduate study or 3 years experience 

GS-7: l.year graduate study or specialized experience as 
controD.ler or pilot 

GS-9: 2 years specialized experience as controller. 

3. Age 

Not over 30 years (no such limitation before 1972). 

4. Reference Checks 

Written inquiries to verify claimed employment and to solicit 
evaluation of candidate on 15 items such as job interest, 
cooperation, and emotional stability. 

5. National Security Check and Investigation 

Security investigation by Government agencies to determine 
suitability for employment. , 

6. Medical Examinations 

• Physical . To detect presence of disabling illness or 
abnormality. Includes a questionnaire on 24 conditions, 
some of which relate to psychological conditions. 

• . Psychological . Applicant cannpletes a psychological 

inventory (Cattell 16 P-F) which yields a profile on 16 
personality factors, e.g., outgoing versus reserved, 
casual versus controlled. 

7. Interview 

Candidate is evaluated in areas of oral expression, poise 
*^^and self-reliance, resourcefulness and decisiveness. Inter- 
view Record and Evaluation Form is prepared by panel con- 
sisting of a controller and a personnel specialist. 
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C. DISCUSSION 

The trainee, then, is an applicant who, having been judged 
eligible by these procedures, accepts an appointment at the GS grade 
level offered to him. The process of selection and appointment is 
not distinguished by its speed, and some candidates have taken other 
jobs by the time an appointment is offered* The numbers of men 
selected for training have fluctuated widely over the ypars* 

Whether each step in this procedure is necessary and whether 
each criterion contributes to selecting controllers who will perform 
well on the job are pragmatic questions • An EPA study v/as commis- 
sioned because the Air Traffic Controller Career Committee (Lhe 
Corson Committee) pointed out in its report. The Career of the Air 
Traffic Controller (Corson et al,, 1970), that selection and training 
were inefficient and expensive procedures • At that time, the attri- 
tion rate was 30 percent during formal training and an additional 20 
percent during on-the-job training (OJT), EPA found that the selec- 
tion procedures could be improved and reported new results on selec- 
tion and training that confirm findings made earlier by CAMI, In 
FY .1974, the failure rate was about 43 percent for en route trainees 
and about 38 percent for terminal trainees. 

The significance of each selection criterion (e^g,, scores on 
the CSC tests, amount and type of experience, maximum eligible age) 
or of any combination of criteria can be determined by what it pre- 
dicts and how well it does so. For example, what is the relationship 
between CSC test scores and ability to complete initial training, or 
to, be qualified as a journeyman controller, or to perform well on the 
job? The relationship between the selection criterion and some later 
measure is called the "validity" of the selection criterion. Among 
the selection criteria, it is known that scores on the CSC aptitude 
tests predict a candidate's ability to complete training. There is 
little relationship between CSC test scores or the other criteria 
used for selecting prospective controllers and success in training 
or performance* on the job. The research which has been accomplished 
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reflects the fact that CSC test scores are readily available for 
analysis, and information about success in training is more readily 
available than information about success on the job. After training, 
one must wait various amounts of time before job performai.re data 
can come into existence. The job performance data, so-called, con- 
sist of supervisors^ ratings which are subjective and generally 
unreliable, at least for statistical purposes, for comparison with 
predictors of success on the job. It is regrettable that little 
effort has yet been made to collect more objective information about 
the performance of journeymen controllers. 

The CARI/CAMI reports* show that individuals with higher CSC 
aptitude test scores show a lower failure rate during training at 
the FAA Academy; they receive higher grades in academic and labor- 
atory work; they also receive higher ratings from their training 
supervisors. Correlations between aptitude test scores and these 
criteria, on which there is much data, rarely exceed 0.35. A cor- 
relation of 0.35 means that CSC test scores predict about 12 percent, 
i.e., (0.35)^, of the variation observed between CSC scores and the 
later measures. A few studies which go beyond training show that 
individuals with higher CSC scores also remain longer with FAA 
(qualified by data for those remaining with FAA up to 7 years). This 
suggests that selection and training actually find, to some extent, 
candidates who are well suited to the journeyman's job. These 
studies, summarized here very briefly, provide the factual basis for 
establishing for prospective controller trainees a minimum acceptable 
score of 210 on the CSC test battery and a maximum age of 30 years. 
Identical prior experience in air traffic work predicts success in 
training. Experience in communications and ground -controlled inter- 
cept (GCI) and as a pilot have little value in predicting performance 
in training, although they are used to justify a higher civil service 
grade at time of entrance. Data have not been collected to support 

"These studies are listed under the names of their authors in the 
references; see Chiles, Cobb, Trites. 
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or reject the validity of the other criteria user" in the selection 
procedure. Undoubtedly, the medical examination identifies signif- 
icant disabilities among candidates, because few controllers leave 
the FAA for medical reasons. 

A better understanding of what controllers do on the job shows 
that new criteria should be added to the selection system. Thus 5 a 
pilot study on ATC trainees by Chiles, Dean, Jennings and West (1972) 
at CAMI shows that a battery of tests which involve skills in 
physical coordination and which also require the candidate to work 
on several tasks at the same time can ppedict supervisors' ratings 
of tx'dinees at the Academy (validations with training in the pilot 
study range from 0.24 to 0.54). Cobb and Mathews (1972) find that 
a new aptitude test called "Directional Headings" correlates 0.41 
with training performance. This test requires the subject to rapidly 
interpret letters, symbols, and degrees in order to determine 
directional headings while being exposed to aural distraction. In 
the Controller Decision Evaluation (CODE) test devised by Buckley 
and Beebe (1972), the candidate judges possible conflictions in a 
simplified air traffic display presented in a sound motion-picture 
film. Education and Public Affairs (1972) shows that a combination 
of psychomotor tests, including CODE, correlates 0.50-0.75 with 
supervisors' ratings. Thus, a case can be made that .selection would 
be improved by including tests of spatial abilities, psychomotor 
performance, and ability to monitor several tasks simultaneously. 

Another possible route for improving selection, with its 
corollary effect on training, is to act on the fact that terminal and 
en route controllers perform somewhat different jobs. This, is ^demon- 
strated in the task analyses performed for FAA by System Development 
Corporation (1971, 1972a-d, 1974a-f). On the basis of aptitude tests, 
training performance, and experimental performance ratings, Trites, 
Miller and Cobb (1965) concluded that en route, terminal, and flight 
service station (FSS) personnel differ in the characteristics required 
for job performance. This is strongly confirmed with a much larger 

135 



ERLC 



13G 



battery of experimental tests in the EPA (1972) study. This shows 
that the inclusion of new tests, including psychomo' ^r ones, could 
increase accuracy of assignment between IFR, VFR, Center, and FSS 
from 25 percent (a random possibility of success when no special 
criteria are used in selecting personnel for these four options) to 
58 percent. Within the IFR and Center options, accuracy levels of 
75 to 80 percent could be achieved. Even better selection should 
become possible when Buckley's work at the National Aviation Facil- 
ities Experimental Center (NAFEC) produces useful objective per- 
formance measures for controllers on appropriately designed traffic 
samples . 

Improved selection procedures should reduce attrition in train- 
ing and attrition due to inappropriate job assignments. Reduced 
attrition can, of course, produce significant reductions in t:he costs 
of training, and this issue is considered in Appendix G. 

The process of selecting controllers includes not only the tests 
and procedures which have been considered here but also the screen- 
ing which occurs as the candidate succeeds or fails at each phase of 
training from novice to full journeyman status. In terms of cost 
and time alone, it is clearly^ preferable to improve selection by 
testing rather than by hiring. For the same reasons, it is also 
preferable to screen personnel at the Academy rather than by OJT. 
Among the various ways of reducing costs and improving the efficiency 
of training, a premium must be placed on improving the selection 
procedures. The evidence addressed in this Appendix is that improved 
selection is possible by adding perceptual and psychomotor tests to 
the battery of selection tests. In addition, candidates should be 
selected by criteria specifically relevant to the ATC specialty to 
which they will be assigned, rather than by the general criteria now 
applicable to all controllers. 
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D . SUMMARY 

The procedure for selecting controllers includes a battery of 
aptitude tests, a medical examination, reference and security checks, 
and an interview. There are also requirements for education, expe- 
rience and age (not to exceed 30 years). To qualify for employment, 
a candidate must meet minimum conditions or scores at each step in 
this procedure. Studies performed for the FAA show that the aptitude 
tests predict, to a modest extent, a candidate's ability to complete 
training and to continue working as a controller. The medical exam- 
ination appears relevant because few controllers leave their jobs 
for medical reasons. There is no evidence to show that any of the 
other criteria used during selection predict success in training. 
There is little evidence to show that selection criteria predict job 
performance, i.e., quality of performance after conpletion of train- 
ing. The inability to determine relationships between training and 
on-the-job performance is due principally to the absence of reliable 
measures of job performance. There is evidence that selection pro- 
cedures could be improved by the addition of a variety of psychomotor 
performance tests, including one which uses samples of simplified 
air traffic situations. Improved selection procedures would be 
expected to reduce attrition during training and thereby save some 
of those expenses. 

E. CONCLUSIONS 

Among the selection criteria presently in use, only aptitude 
test scores predict success in training (to a modest extent), while 
the maximum age standard appears to reduce attrition after training. 
No significant relations have been demonstidted between success as a 
controller and either the initial selection criteria or performance 
during training. Significant improvements in selection and reduced 
training costs are feasible by incorporating psychomotor and con- 
troller-like tests into the selection battery. 
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A, INTRODUCTION 

The purpose of this Appendix is to review current methods of 
measuring the performance of air traffic controllers. The review is 
based on descriptions of the present procedures, many studies of air 
traffic controllers conducted by the Federal Aviation Administration 
(FAA), and a large background of knowledge about performance measure- 
ment in systems, both simulated and operational in nature, with func- 
tions identical or similar to those in the FAA air traffic control 
(ATC) system. 

Performance measures are very important to the FAA because they^ 
provide the means needed, together with other factors such as cost 
and flexibility, to evaluate the ultimate effectiveness of alternative 
methods of training air traffic controllers. Performance measures 
are also needed for many other purposes of interest to the FAA such 
as, for example, determining the controller's maximum useful work- 
load, the distribution of traffic loads among sectors, the impact of 
new or proposed types of ATC equipment, and the significance of 
various tests and criteria used for the selection of controllers. It 
is a matter of some interest that objective performance measures are 
not in use at present, although their feasibility has been demonstrated 
in experimental work conducted at the National Aviation Facilities 
Experimental Center (NAFEC) (Buckley, O'Connor and Beebe, 1969). 

Objective performance measures would describe a controller's 
performance in such objective terms as, for example, the number of 
aircraft handled, the average delay per aircraft instruction, and 
the number of control instructions and average communication times — 
in each case, probably as a function of traffic load. Measures of 
this type must be contrasted to subjective rating schemes, "/such as 
over -the -shoulder evaluations, which are presently in use and which 
will be discussed below. 

The ingredients needed to develop objective measures of the 
controller's performance have been present in FAA for a long time. 
Task analyses, needed to describe precisely what the controller does 
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on the job and thus to train him appropriately and to evaluate his 
performance afterwards, were developed by Nagay (1949, 1950) and by 
Courtney and Company, 1960a-c). The duties of a controller have i 
also been described, more or less precisely, in many other reports, 
e.g., Jacksonville Air RouttB Traffic Control Center (ARTCC) (1970), 
Civil Service Commission (1968), Arad (1964), Warskow, Hooton and 
Burns (1969), Kuprijanow (1970) and Ratner et al. (1972). However, 
there has been a general lack of detailed, up-to-date job information 
about what the controller actually does until the recently completed 
work by System Development Corporation (SDC, 1971, 1972a-d, 1974a-f). 
This extensive effort,, which is based on detailed and verified des- 
criptions of the work performed by controllers, will be discussed 
later in this Appendix. 

The development of objective performance measures at FAA can be 
seen in the work of Anderson and Vickers (1953) and Vickers (1959) at 
Indianapolis, Pearson et al. (1965) at Oklahoma City, and Buckley 
et al. (1969) and Buckley and Beebe (1972) at NAFEC. Closely related 
work on air traffic control, concerned with both training on system 
simulators and the objective measurement of performance was done at 
Ohio State University in 1952-1961 for the Air Force, at the Mitre 
^ Corporation for the FAA in 1969-1973 and at the Rand Cprporation and ^ 
SDC on the SAGE System for the Air Force, starting in. 1952. A history 
of these interesting and important efforts, including many FAA stud- 
ies not cited here, may be found in Parsons (1972). Despite the 
evidence of extensive previous efforts at FAA and elsewhere, objective 
performance methods are not in use now'fiy FAA. 

B. SUBJECTIVE PERFORMANCE MEASURES 

At present, the FAA assesses the performance of air traffic con- 
trollers in several ways. Trainees are judged on the basis of grades 
achieved on academic tests and laboratory exercises. During on-the- 
job training, there are written examinations on rules and procedures 
and over-the-shoulder evaluations by the supervisors. The ultimate 
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performance measure for the developmental 's training is his pass or 
fail status. As a journo^man, the controller is assessed for pro 
ciency by periodic written exams, over-the-shoulder evaluations, and 
the Employee Appraisal Record (EAR), required by the Civil Service 
Commission . 

The explicit purpose of these tests is to detennine the profi- 
ciency of the developmentals and the journeymen, but not to grade 
their performance. Besides the essential purpose of qualification, 
the various tests and evaluations are used largely for diagnostic 
purposes so that remedial training may be specified as required, and 
the controller reexamined accordingly. In the extreme case, of course 
the developmental or journeyman may be eliminated for an unacceptable 
performance. But the criterion is still one of pass or fail. 

Except for determining whether remedial training is needed, this ' 
method of measuring performance .does not establish the level or 
degree of proficiency which distinguishes individual air traffic con- 
trollers. Although it would be difficult to believe that all air 
traffic controllers are equivalent in perfonnance, the use of "pass 
or fail" performance measures makes it appear that this is indeed the 



case, 



The use of categorical performance measures also makes it diffi- 
cult to establish whether one method of training is more effective 
than another, even if it is assumed that the measures are reliable and 
valid. The essential reason is that these measures have no dispersion, 
i.e., normal distribution in the statistical sense. In effect, such 
measures cannot be used to detect differences between alternativR 
methods of training because, according to the measures, most students 
appear to perform equally well, although this cannot be the case. As 
for those who fail, we do not know by how much. The limitation is 
due, obviously, to a matter of policy which determines whether defic- 
iencies exist and not whether there are quantitative differences in 
levels of performance. Another consequence of this policy is that the 
training system is inefficient if it trains some men longer than is 
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really required. That is because training is oriented to provide the 
required number who can just qualify. 

However, the major issue is whether the pass-fail grades, EARs, 
and over-the -shoulder evaluations are suitable measures of an air 
traffic controller's performance. There are several reasons to be- 
lieve that there are significant limitations to the present use of 
ratings in general, and to the over-the -shoulder rating in particular. 
These relate to the subjectiveness of the rating procedure and to the 
iack of standardization in the traffic samples used to evaluate a 
controller's performance. 

It is of interest to note that this point has been made strongly 
within FAA, but to little noticeable effect, by a Southwest Region 
staff study of the training program (Curliss et al., 1974, p. 25): 

The reason for having a technical training program is 
to meet an operational need. The test of the training lies 
in the degree that personal skills are acquired or enhanced 
through that process. Performance on the job is the tinal 
and best measure of the effectiveness of training. 

Unfortunately, the FAA has yet to devise reliable ob- 
jective measures of the controller. The "over-the-shpulder" 
evaluations required by the TPAP program are at best sub- 
jective judgments by peers and supervisors using varying 
standards. The a|)sence of objective, relevant measures of 
individual performance makes it difficult to know what im- 
provements are truly required. 

It may or may not be obvious that the over-the -shoulder eval- 
uation is, at its heart, a subjective procedure. What "subjective" 
means is that this method of evaluation represents one person's view 
of another, i.e., the supervisor's judgment of how well the controller 
performs his job. Supervisors often differ in their appraisal of an 
employee for a variety of reasons such as, for example, differences 
in the extent of their knowledge about the employee's performance, 
different standards concerning the nature of an adequate performance, 
and because social and personality factors may intrude in a subtle 
way into the judgment of on-the-job performance. This evaluation has 
an "absolute" quality only as long as it is made by one supervisor. 
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Its subjective nature would be demonstrated directly if journeymen 
were evaluated independently at the same time by two or more super- 
visors, rather than by one. The FAA procedure reserves over-the- 
shoulder evaluation of each controller to the one supervisor who knows 
him best. 

The extent to which subjective factors may influence over-the- 
shoulder ratings of controllers by supervisors is not known directly, 
but it can be estimated from studies conducted for FAA and its 
predecessors since 1950, These are studies in which performance 
ratings of a controller have been reported on more than one occasion. 
The agreement between the ratings provides an estimate of the reli- 
ability (or repeatability or consistency) of that method of eval- 
uating pf.irf ormance. Most of the studies use a supervisor's rating 
form and are subjective in nature, but a few have used objective 
measures, such as number of aircraft handled, number of delays, and 
the like. 

Nagay (1949, 1950) developed a rating form on which supeWisors 
identified significantly effective or ineffective behavior observed 
while evaluating a controller's performance.* The form was tested 
experimentally by 48 senior controllers* who Qt>served 42 controllers 
at work in the New York, Washington, and Chicago Centers. At that 
time, the method of control was nonradar. The nature of the exper- 
iment permitted comparison between supervisors' ratings of one con- 
troller, i.e., estimates of the reliability of the ratings, as 
follows : 

Reliability 
Number of (Correlation) 
Comparison Comparisons Observed Adjusted 

Between supervisors' ratings 80 0.43 0.94 

at the same time 

Between supervisors' ratings 156 0.22 0.85 

at different times 



This account is based on information in Education and Public Affctiis, 
Inc. (1970), pp. 19-20, and Buckley, O'Connor and Beebe (1969), 
pp. 1-1 to 1-2. 
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The observed correlations are based on a limited number of observa- 
tions, i.e., controllers worked in 12 watches, and each .senior con- 
troller made only an average of 6.5 observations. It is possible to 
estimate what the reliability coefficient would be if more observations 
were made. TKe adjusted reliabilities, above, show what would be 
expected if the controllers were observed continuously for one year 
(237 vatches). Of course, no one would realistically argue that 
supervisors could or should spend so much time observing controllers 
in order to generate reliable evaluations, but it should be obvious 
that reliable ratings would require an extensive series of observa- 
tions . 

Trites, Miller and Cobb (1965) summarize various studies con- 
cerned with job performance measures for controllers conducted at 
CARI from 1961 to 1965. The job performance data consisted of ratings 
made by supervisors on controllers on 16 items, such as "ability to . 
understand and apply controller procedures", "display of good judg- 
ment", and "demonstrated aptitude for air traffic control activities". 
The ratings were on a five-point scale, with "excellent" and "unsat- 
isfactory" at opposite extremes. The reliability (correlation) of 
ratings made by supervisors in various studies was as follows: 



Group N r 

En route controller 468 0,58 

Terminal controller 262 0.78 

En route controller 367 0,43 

Terminal controller 244 0.47 



The two higher reliabilities are for ratings made at about the same 
time; the two lower reliabilities are between the earliest and the 
most recent ratings of the same individuals. From the text, it ap- 
pears that "at about the same time" signifies within about two 
months; the time interval for the smaller reliabilities is not speci- 
fied. 
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In a .paper concerned with attrition-retention rates of air 
traffic controllers during 1960-1963 and 1968-1970, Cobb, Mathews 
and Nelson (1972, p. 1) comment upon the reliability of FAA perform- 
ance-rating methods as follows: 

Moreover, experimental performance-rating procedures 
were developed and employed in .several follow-up 
studies (references given) because the officially 
derived ratings (e.g., those based on the "Employee 
Appraisal Record," and others which were rendered 
primarily for remedial or diagnostic purposes) offered 
little potential for individual differentiation. The 
experimental ratings submitted by supervisors, crew 
chiefs, and peers of the subjects were somewhat less 
"haloed" than those rendered for official purposes but, 
like the latter, they did not possess a very high 
degree of reliability. 

Cobb (1968) reports a survey of 568 controllers at four ARTCCs, 
of whom about half were rated either two or three times. Rating 
Form C contained 33 items which were verbatim copies of the perform- 
ance indicators or appraisal standards specified in the semiannual 
over -the -shoulder ratings of each controller. Form B contained 14 
items of a more general and less technical nature than Form C. 
Reliability, based on the average intraclass correlations between 
the several ratings, ,was: ^ 

Form B 0.35 
Form C 0.29. 

Cobb believes that the low reliabilities may be due to the fact 
that although many of the ratings were made by the controller's im- 
mediate supervisor, other supervisors participated who may have had 
less direct knowledge of the controllers. Cobb and Harshaw (1967, 
unpublished) were able to find EARs (Employee Appraisal Records, 
Form 3693) that gave findings similar to those of the Form B and Form 
C ratings with agp and length of experience. Cobb and Harshaw say 
that supervisory ratings of performance rarely attain a high degree 
of reliability. 

Cobb, Nelson and Mathews (1973) examined several types of per- 
formance evaluation ratings collected on 614 terminal area traffic 
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controllers at 17 terminal facilities. The experimental Air Traffic 
Control Specialist (ATCS) Performance Evaluation Form contained 29 
aspects of performance for evaluation, very similar in content and ^ 
wording to the over-the-shoulder rating; eight of the items came 
from an. experimental form developed by CAMI relating to teamwork, 
tactfulness, and adaptability. Ratings were made, variously, by 
coworkers, crew chiefs, and supervisors. The following findings 
relate to reliability: 

Comparison 

Between coworkers 

Between crew chiefs 

Between supervisors 
(ratings on 121 
controllers) 



"^Based on z-coef f icient transformation. 

The new controller performance rating procedures developed by 
System Development Corporation (1971, 1972a-d, 1974a-f) have been 
reserved for separate treatment. These incorporate significant 
features which would probably improve the reliability of performance 
ratings, although the extent to which this may be true cannot be de- 
ter^mined from the various reports about the procedures' development 
and field tests. The SDC report was submitted recently, and the pro- 
cedures it proposes have not been implemented at the present time 
(December 1974). 

The SDC performance rating system consists of Over-t he -Shoulder 
Training Reviews and Extended Performance Ratings for en route and 
terminal controllers, and it may be summarized briefly as follows: 



Range of 
Correlations 

0.21-0.47 
0.40-0.54 
0.62 



Average ^ 
Correlation 

.0.39 
0.46 
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Purpose 

Suggested Frequency 

Number of Items 
Rated 

Basis of Review 



Duration 



Scoring 



Results 



Over-the-Shoulder Performance 
Review 

Identifies need for addi- 
tional training. 

Four reviews a year, 

27 in 7 skill areas. 

Specific observations when 
traffic is adequate (moder- 
ate or higher). 

Four 10-min segments for 
each review. 

Dots identify the areas in 
which the controller is 
deficient. 

Number of deficiencies, i,e,, 
areas in which additional 
training is required. 



Extended Performance Rating 

Determines level of profi- 
ciency. 

Twice a year. 

73 in 4 skill areas. 

Review of over -the -shoulder 
and other data. 



Numerical, Rating from 1 
to 7 for proficiency on 
each item; from "unaccept- 
able" to "excellent." 

Profile; average of above 
ratings in four areas plus 
overall rating. 
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The method of performance evaluation developed by SDC incorporates the 
following significant features: 

• The controller's job has been determined through direct 
observation of the way in which controllers perform their 
duties at the present time. The results are contained in 
a series of detailed job analyses and flow diagrams not 
previously available. 

• Controllers are rated in those aspects of their job duties 
which are regarded as significant for effective performance 
and on which their behaviors are observable. These job 
duties were identified jointly with controllers and modified 
in a series of trial applications. Manuals were developed 
to explain how the ratings are made, to define terms, to 
show hov^' scores are developed, and so on. 

• The conditions of observation are more carefully defined and 
are more standardized than in previous rating procedures 
used by FAA. This applies to number and duration of 
observations, peak traffic count and degree of complexity, 
and training of observers. Minimum traffic requirements 
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were specified numerically in the preliminary version but 
not in the final one. Attention is given also to discus- 
sing the results of the evaluations with the controllers. 

These methods of rating are more carefully specified but are 
not necessarily less subjective than those followed previously. To 
the extent that the controller would be evaluated on behaviors which 
are directly observable, which are more carefully described for con- 
sideration by the raters, and which are truly relevant to job per-^ 
formance, one would expect improved reliability, i.e., better agree- 
ment between supervisors making the ratings. 

The minimum length of observation is given as four 10-minute 
periods. Although longer periods of observation tend to increase 
the reliability of ratings, the selection of a 40-minute total length 
appears to have been done arbitrarily. The summary report indicates 
that there was significant agreement between raters, i.e., high reli- 
ability, but no data are presented (SDC, 1974a, pp. 25, 31, 35, 51). 
The overall inter-rater reliability for tests run earlier at two 
locations was 0.65 and 0.67 (private communication from D. L. Dickson, 
SDC, 9 September 1974). Inter-rater reliability for the final over- 
the-shoulder pDrocedure was not determined, in accordance with guide- 
lines received from FAA (SDC, 1974a, pp. 66-67). 

One of the major reasons for the low reliability of over-the- 
shoulder ratings is variability in the conditions of observation, 
which includes such factors as the density and complJexity of the 
traffic sample being observed and the duration of the observation 
period. Buckley, O^Connor and Beebe (1969) used the NAFEC simulator 
to present traffic samples which differed in levels of density from 
6 to 12 aircraft under simultaneous control. A total of 36 journey- 
man controllers worked two runs at each of three levels of traffic 
density. Traffic was built up in a 15-minute period which preceded 
each run. Three controllers, specially trained as observers for this 
experiment, independently rated the performance of each controller on 
each run. 
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TUxe median reliability in observing the same man on the same 
run was 0.S3. Using average ratings, the median reliabilities of 
nine types of observer ratings at three different traffic densities 
were O.yS, 0.70 and 0.56; reliability decreased as density increased. 
The agreement between judges in judging conflictions was 0.83. 

Buckley, O^Connor and Beebe (1969) show a higher reliability for 

performance ratings than those found in other FAA studies because of 

the control they had over the traffic samples and because of the 

training their observers received. Excluding their data 5 the median 

reliability is 0.43 for other FAA studies on performance ratings 

which appear in this Appendix. Since the magnitude of variance 

2 

explained by a correlation is r 5 this accounts for about 18 percent 
2 

(0.43 ) of the relationship between supervisors^ ratings; the remain- 
der remains unexplained or uncontrolled. Though these reliabilities 
are regarded here as being low? they are about as high as one finds ^ 
for subjective rating scales. Thus, improved performance evaluation 
schemes must be sought by appealing to methods of measurement other 
than subjective rating scales. 

C. OBJECTIVE PERFORMANCE MEASURES 

It is feasible to measure a controller's performance in an 
objective way, that is, in a way that produces quantitative scores on 
various characteristics of an operator's performance. Measurable 
performance characteristics of major interest include speed and 
accuracy of response, maximum number of aircraft handled, maintenance 
of separation standards (measured in time and distance), and the like. 

The primary device needed to develop objective performance 
measures is one which records the flight histories of aircraft and 
the actions taken by the controller to direct their progress. The 
capability of recording this information already exists in the 
National Airspace System, Stage A (NAS-A) and it will exist in the 
Automated Radar Terminal System III (ARTS III), when Package One, 
Enhanced Target Generator (ETC.)., is introduced. At present, howevei^', 
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these records are used primarily for diagnostic purposes and not for 
performance measurement. Performance could be recorded simply by a 
TV camera with recording tape. In this case, analysis of the record 
would be accomplished, in a most tedious manner, at a later time. On 
the NAFEC ATC simulator, the recording, data processing, and reporting 
of performance measures are now being accomplished virtually in real 
time. Similar on-line data processing functions will be performed in 
future ATC systems which monitor traffic automatically and which give 
warnings of potential conflictions . Air traffic control simulators 
provide the means for repeated observations of performance on traffic 
samples with known characteristics. Most (but not all) simulators 
include some means of recording performance. Finally, objective per- 
formance measures and ATC simulators provide the means, by observing 
the actions of many controllers on the same traffic samples, of estab- 
lishing statistical norms for evaluating the performance of controllers 
in an objective manner in precisely controlled situations. 

Objective performance measures, no matter how exquisite, would 
not be useful for evaluating some significant characteristics of a 
controller.' s performance such as phraseology, leadership qualities, 
and participation in teamwork. Subjective rating schemes would still ~" 
be needed to evaluate these attributes. 

Simulators capable of measuring the performance of air traffic 
controllers have been used by the FAA for many years. Pearson, 
Hunter and Neal (1965) describe a radar air traffic simulator at CARI 
useful for research on training, selection, controller proficiency, 
workload as a function of targets and speed, display design, and the 
like. Target motion is derived from film strips which advance one 
frame every 10 seconds. Individual control of each aircraft via a 
pseudo-pilot is not possible with this equipment. Scoring was limited 
to the speed and accuracy of response to key events programmed on 
selected frames, e.g., entries of new aircraft into the sector, inter- 
section crossings, airway 9^anges, voice transmissions, and hand -of fs. 
The Pearson report describes preliminary experiments which show 
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different rates of learning radar control by terminal and flight 
service station (FSS) personnel. Reliability data are not reported. 
No reports showing further use of this device have been found. 

The NAFEC simulator used by Buckley, O'Connor and Beebe (1969) 
has much greater capabilities than the one described by Pearson, 
Hunter and Neal. The pioneering work of Buckley, O'Connor and Beebe 
clearly demonstrates that it is feasible to measure the performance 
of an air traffic controller in an objective anu reliable way. Here, 
some of the major results of their unusually comprehensive "prelim- 
inary" investigation are noted. This work is still going on, and the 
current NAFEC simulator is improved over that used in the initial 
study. 

The NAFEC Air Traffic Control Simulator (Model A) used by Buckley 
can present up to 48 aircraft, each controlled separately by a "pilot" 
who can communicate with the controller via "radio" and who can alter 
the aircraft's speed and flight path. The simulated airspace contains 
radars, beacons, and communication equipment. The performance of 
four controllers can be recorded separately on the following basic 
measures : 

• Number of conflictions 

• Number of delays 

• Delay time 

• "Number of aircraft delayed 

• Aircraft time in system 

• Aircraft time in system for completed flights 

• Flight time deviation for completed flights 

• Number of completed flights 

• Number of control instructions 

• Number of contacts 

• Communication time 

• Number of aircraft handled. 

Since this was an exploratory investigation, other data were 
also collected such as heart rate and galvanic skin response, 
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biographical data, experimental performance ratings in the field 
from a previous study 5 scores on a personality test, the most recent 
Employee Appraisal Records (Civil Service ratings), and controller's 
opinions. Three observers independently rated each controller's per- 
formance on 27 measures. The ratings on all 27 measures proved to be 
so voluminous that ratings on only nine measures were analyzed. Data 
on another type of simulation, called Controller Decision Evaluation 
(CODE), were also collected and will be described below. 

A significant effort was given to developing samples of traffic 
which were representative and which increased in difficulty so that 
all controllers could handle the easiest, while the most difficult 
samples would probably overload the best controllers. A total of 36 
journeyman controllers was tested twice at each of four levels of 
density for a total of eight runs containing 340 flights. Each run 
lasted 1 hour, not counting a 15-minute period during which traffic 
i)uilt up. Since each controller was tested twice on the same 1-hour 
task, this provided the basic data needed to determine reliability^' ''^ 
i.e., the repeatability of vat^ious measures at different levels of 
traffic density. This study is the first one to make such a determin- 
ation. 

The data were analyzed to determine the statistical significance, 
if any, of various combinations of the basic measures at various 
levels of traffic density, e.g., number of conf lictions/number of 
aircraft in sample, and so on. One would expect that some but not all 
measures or combinations of measures would be found to be significant. 

The following findings are particularly significant for the 
prospect of developing reliable and objective performance measures: 

• The median reliability of all objective measures was 0.65. 
Reliability increased from about 0.60 at low traffic density 
to about 0.70 at high density. This trend is contrary to the 
reliability of observers' ratings, reported earlier in this 
Appendix, which decreased from 0.58 to 0.39 as density in- 
creased . 
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• Multiple correlations of about 0.45 were found between per- 
formance measures in the simulator and various combinations 
of job performance ratings made previously by supervisors 

of these controllers, peer nominations, and observer ratings 
made during the study. This is taken to mean that simulator 
scores have some validity, i.e., that they produce scores not 
unlike those provided by presently accepted methods. 

• A factor analysis was undertaken to identify the least number 
of common elements needed to account for the variability in 
the great number of measures used in this experiment. Four 
dominant types of factors, with these key elements, emerged 
from the statistical analysis: 

1. Delay-Volume Factors 

- Expedition of high density traffic 

- Minimization of delays, low and moderate density 
traffic 

- Expedition and maximization of com'^leted flights, 
moderate density 

- Random high average delay time, low density 

2. Separation-Confliction tdctors 

- Failure to separate aircraft, low and moderate 

densities j 

- Violation of separation, high density 

- Random high density conflictions 

3. Communications Factors 

- Frequency vs. duration of contacts 

- Volume of control instructions, low and moderate 
densities 

- Transmission time 
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4. Non -Simulation Factors 

- Experience 

- Heart rate 

- Supervisory rating 

- Current age and age at entry on duty. 

The following performance measures are recorded for use in 
further studies: 

• Conflicts per aircraft handled 

• Conflicts per delay 

• Delays per aircraft handled 

• Delay time per aircraft scheduled 

• Aircraft time in system per aircraft scheduled 

• Proportion of complete flights schedule actually completed 

• Contacts per aircraft handled 

• Communication time per contact 

• Proportion of aircraft scheduled that were handled. 

The Buckley study clearly demonstrates that it is feasible to 
develop objective performance measures for air traffic controllers, 
and the study has performed the important service of identifying the 
most promising measures from a much larger number of candidates. 

Equally important are two conditions which must be satisfied in 
order to measure the performance of controllers: (1) standard traf- 
fic samplf^<^ constructed with specified levels of difficulty. based on 
such features as density, angles of convergence between traffic lanes, 
number of hand-off s, proportion of climbing and descending aircraft, 
and mix of aircraft performance characteristics in the sample; and 
(2) extended periods of observation, probably on the order of two 
1-hour runs for each test. 

The Controller Decision Evaluation (CODE) test, which is in- 
cluded in. the study by Buckley, O^Connor and Beebe (1969), does not 
use the NAFEC Model A simulator. In the CODE test, the controller 
observes a radar display which shows the positions of aircraft along 
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airways. The scope displays come from a previously prepared film 
strip. The controller also receives information via a headset and 
keeps flight progress strips and aircraft identity up-to-date as new 
frames appear. The controller's task is to indicate when action 
should be taken to avoid any possible confliction that may arise. 
Each run contained six or seven conf lictions . Each subject was tested 
twice at each of three densities of traffic, for a total of six runs. 
The basic Jata for each controller were the number of conflictions 
correctly detected, and the false conflictions, i.e., reports of con- 
fliction when there were none. Because of problems with the equipment 
only 18 of the 35 controllers were tested, and it appears that the 
runs were not long enough. Under these conditions, the following 
repeat reliabilities (between runs at the same level of density) were 
found : 

Level of Density 
2 3 4 

Number of correct detections 0.24 0.43 0.63 

Number of false positives 0.54 

Certain CODE scores showed some significant correlations (about 
0.70) with some of the performance measures obtained in the dynamic 
simulations on the Model A simulator. The areas of high correlation 
suggest that CODE measures assess a controller's ability to forecast 
the ^^^ture positions of the various aircraft under observation. In- 
creased delays, as found for some controllers, appear to result in- 
directly from an inability to forecast potential conflictions rather 
than directly as a result of higher volumes of traffic. At present, 
these findings are incomplete and only suggestive at best. 

The CCDE test was modified for projection by motion picture film 
v.'ith a sound track (Buckley and Beebe, 1972). Instructions for taking 
the test are also on film. The new films were made with the NAFEC 
Digital Simulation Facility which gives an alphanumeric tag to air- 
craft and a table, updated as required, with altitude, speed,, and 
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route data. As previously, the controller is supposed to identify, 
as soon as possible, pairs of aircraft that may violate separation 
standards. The modified test can be administered simultaneously to 
groups of men (ten in this study). In a study of 19 controllers, the 
median reliability was 0.75 (ranging from 0.60 to 0.90 for five 
measures tried here). 

The obvious advantage of the CODE test is its ease of adminis- 
tration. Presumably, traffic samples of various levels of difficulty 
could be developed for testing at various stages of training, for 
illustrative purposes in connection with training, and, if not too 
complicated, as a selection test for prospective controllers. It is 
not inconceivable that it could also be developed into an instrument, 
with new versions from time to time, to test selected aspects of the 
proficiency of journeyman controllers. 

Buckley's current work at NAFEC is directed towajL-'d the develop-' 
ment of a Controller Performance Measurement (CPM) test package which 
would include procedural instructions, scoring methods, and normative 
data (NAFEC Agreement No. 21-254, dated 22 June 1973). This is a 
research and development effort planned to continue over several 
years. An improved CODE test, intended for validation of selection 
procedures, will also be developed 'onder the program. Progress 
reports are not yet available at the time of this writing (December 
1974). 

D. SUMMARY 

Present procedures for evaluating the performance of controllers 
center around the over-the-shoulder evaluation, are subjective, and 
show low consistency on repetition, i.e., low reliability. FAA 
studies, wnich are still continuing at NAFEC, show that the perform- 
ance of air traffic controllers can be measured in an objective way. 
The objective measures show a higher reliability than the subjective 
ones. For performance measures to be reliable, it is necessary to 
observe controllers performing on samples of air traffic carefully 
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standardized for level of difficulty, i.e., density, complexity, and 
potential conflictions. For reliability, it also appears necessary 
to observe a controller's performance over reasonably long periods of 
time, so as to provide a measure which is the average of at least two 
1-hour periods of observation. Objective performance data are needed 
to evaluate the effectiveness of various methods of training. They 
would also have great value in selecting controllers, establishing 
qualification standards, evaluating the proficiency of developmental 
and journeyman controllers, determining v^ontroller workloads at 
various levels of traffic, and thereby contributing to the design of 
en route and terminal sectors, 

E. CONCLUSIONS 

Studies at NAFEC have shown that the performance of controllers 
can be measured in an objective manner. These measures show a higher 
reliability than the over-the-shoulder rating methods currently in 
use. Objective performance measures are needed to assess various 
methods of training controllers, the proficiency of controllers, and 
optimum workloads, and for similar applications concerned with the 
overall efficiency of the air traffic control system. 
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A. INTRODUCTION 

The purpose of this Appendix is to describe briefly the present 
training programs^ for centers and terminals, and to consider the 
relevance of the curricula to present operational practices, standard- 
ization in methods of training among the Regions > and ways of im- 
proving the training program. Information used for this purpose 
comes from many documents and reports of the Federal Aviation Admini- 
stration (FAA), visits to the FAA Academy and field installations, 
and from a survey of training at field facilities based on analysis of 
answers to a questionnaire prepared by the Institute for Defense 
Analyses (IDA). 

B. TRAINING A CONTROLLER 

This section describes the training program which transforms an 
apprentice controller into a fully qualified journeyman in a period 
of four years. The training of en route and terminal specialists is 
considered here; the flight service station option is not considered. 
The selection procedure is described fully in Appendix D, which also 
suggests that trainees should be assigned to the en route or terminal 
option on the basis of their test scores, a procedure not being fol- 
lowed at present. The typical trainee is a high school graduate, not 
older than 30 years, who achieves a score of at least 210 on the 
Civil Service Commission (CSC) air traffic control (ATC) aptitude 
test, and who is rated as a GS-7. Only 40 to 50 percent of all appli- 
cants attain the qualifying score (Cobb and Mathews, 1972). Because 
of relevant experience in previous employment, about 10 percent of 
the^ applicants, are given GS-9 ratings. In a special program for dis- 
advantaged groups, a limited number of individuals are given a GS-4 
or GS-5 rating and 17 weeks of predevelopmental training at the FAA 
Academy before entering the regular developmental training program. 

1. En Route Option 

En route ATC training is conducted in three phases (FAA, 1972). 
Phase I, called Indoctrination and Pre -Control, is conducted at en 
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route facilities. The approxdjnate course time is 340 hours, and 
programmed lesson plans developed at the FAA Academy are used. In 
Phase I the developmental specialist learns about the air traffic 
control system, the functions of centers, towers, and flight service 
stations, the regulations, the navigational aids, and the communica- 
tions used in air traffic control. He is also taught FAA personnel 
policies. He learns to perfom, under supervision, such air traffic 
control functions as drawing area maps, processing flight plans, dis- 
tributing flight progress strips, encoding and decoding weather in- 
formation, operating the communications switching system, entering 
messages into the computer, and transmitting flight plans and esti- 
mates. Associated with all these "doing" capabilities is a long list . 
of "knowledges" that the trainee must acquire. 

When nearly a year has elapsed, the developmental specialist 
becomes eligible to move up a grade and to initiate Phase II train- 
ing. Before entering Phase II, he takes 120 hours of training in 
prefamiliarization, which must be given in the eight -week period 
immediately preceding entry into Phase II training. In this course 
he learns about strip marking and about the synthetic area sectors 
and letters of agreement that will be used for simulated environmental 
training. He must be able to work low- complexity demonstration exer- 
cises on the simulated environmental sectors. 

Phase II is entitled Radar and Non-Radar Control. The non-radar 
control training course is conducted at the FAA Academy as "well as at 
the en route facilities. Each Region determines where its develop- 
mentals will be" trained. The course includes procedures laboratory 
preparation and simulated environmental training in a non-radar lab- 
oratory. The length of the course i6 approximately 304 hours. This 
training permits a developmental specialist to perform under immediate 
supervision as a member of a sector team on two sectors in his as- 
signed facility, to record clearances and control information on 
flight strips, issue clearances, record control information, and 
perform other tasks. 
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Phase II radar control is supposed to be taught at the FAA 
Academy after completion of the non-radar control course* It takes 
about 240 hours. However, since the Academy does not have adequate 
radar simulation equipment, this training is given at the National 
Aviation Facilities Experimental Center (NAFEC) or^in the operational 
environment of the facility. The trainee is taught to align and 
adjust radar equipment, interpret the radar display, hand off air- 
craft to the next sector, operate the radar sector, and provide 
traffic advisories on request. 

Phase III, Sector Qualification Training, is conducted by the 
facilities as on-the-job application of previously learned skills and 
knowledges utilized for the purpose of qualification in the specific 
sectors to which the developmental will ultimately be assigned. A 
maximum of 60 hours of training is given for each position for which 
qualification is required. The specialist must demonstrate his 
ability to control traffic by the use of applicable radar separation 
standards, procedures, and techniques in a variety of situations. 
After this training is completed, he will be able to perform inde- 
pendently all the functions of a full-performance (journeyman) con- 
troller, under general supervision as^ a member of a sector team on 
the required number of sectors in his assigned facility. 

2. Terminal Option 

Terminal controllers enter the system in the same way as en route 
controllers and follow the same Civil Service procedures. After 
selection, they are given Phase I indoctrination and orientation in 
a 40-hour course of instruction at a facility. When this training is 
completed, the trainee will be able to describe the air traffic con- 
trol system, the functions of centers, terminals, and flight service 
stations, applicable Federal Air Regulations, navigational aids, and 
canmunications, as well as FAA personnel policies. 

Following the completion of Phase I, the trainee is sent to the 
FAA Academy where he is given Phase II, called Pre-Control. This 
260-hour course teaches the trainee to operate the interphone system, 
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operate flight data entry and printout equipment, prepare and dis- 
tribute flight data strips, receive and relay weather information, 
and perform a number of other tasks as noted in the Air Traffic 
Training Handbook (FAA, 1971a). After the training at the Academy is 
completed, the trainee returns to his assigned facility for position 
qualification. 

Phase III, Visual Flight Rules (VFR) Control, requires 400 hours 
of training at the facility. Of this time, 160 hours is devoted to 
procedures training and 260 hours to position qualification. When 
this training is completed, the trainee will be able to perform 
ground control and local control functions independently under gen- 
eral supervision. 

Phase IV, Instrument Flight Rules. CIFR) Non-Radar Control, con- 
sists of two parts. Developmental procedures is a 240-hour course 
conducted at the FAA Academy. Position qualification requires 80 
hours per position and is conducted at the facility. Upon completing 
this training, the developmental specialist should be able to control 
IFR non-radar traffic at a complexity factor equivalent to 15 oper- 
ations per hour, under immediate supervision. After position quali- 
fication, he should be able to perform under general supervision all 
functions of non-radar control at his assigned facility. 

Phase V consists of 200 hours of developmental procedures train- 
ing and 80 hours of position qualification for each radar position. 
Procedures, training is taught at the Academy, and position quali- 
fication is taught at the facility. After completion of training, 
the specialist should be able to control traffic using radar sep- 
aration standards, procedures, and techniques. As a member of a 
traffic control team, the specialist should be able, under immediate 
supervision, to control traffic at a complexity factor equal to 25 
operations per hour. After completing facility position qualification 
the specialist should be able, under general supervision, to perform 
all radar controller functions as specified in the Air Traffic Train- 
ing Handbook and as modified to meet operational requirements unique 
to the facility to which he is assigned. 
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In a recent evaluation of the Academy^ s training programs, 
instructors were sent from the Academy to a number of facilities. 
The following problems were noted:** 

• The existing radar simulation is unsatisfactory, and the 
capabilities of Automated Radar Terminal System (ARTS) III 
are needed for training purposes. 

• The centers" do not have proper radar training; they do not 
use the Academy curricula but rather rely primarily on on- 
the-job training (OJT). 

• The predevelopmental training program to be discussed in the 
next section is turning out better controllers than the 
regular program. 

(No data were provided to support these observations, however.) 

C. PREDEVELOPMENTAL TRAINING 

The FAA has been conducting a program for individuals who cannot 
be hired at the GS-7 entry grade and are thus ineligible for entry 
into the developmental training program. They enter, instead, into 
the predevelopmental training program, which is aimed at assisting 
members of minority groups to enter the ATC career field. A 17-week 
course at the Academy and post -Academy training in the field facili- 
ties give these individuals fundamental knowledge of aviation and 
the National Airspace Systtjju, Individuals in this program enter on 
duty at grades of GS-4 or GS-5 and, upon completing training and 
meeting minimum time-in-grade requirements, are promoted to GS-7 and 
enter the developmental training program. 

Two aspects of the training need some comment here. First, the 
program does not fully reach the people for whom it was designed. 
Many in the program are either not from minority groups or have sound 



Based on a discussion with the Evaluation Officer at the FAJl Academy, 
May 7, 1974. 



157 

164 



educational backgrounds and would appear not to be in need of this 
type of training. Thus, many individuals enter the service through 
the cellar door, so to speak, and while they enter at a lower grade 
and therefore lose some time and :>ay, they do get into the system* ' 

The second aspect of the program has to do with its success. It 
generally appears to be successful, in that it does get members of 
minority groups into the service, and they seem to be performing 
capably. What is not yet known is how well these people are being 
accepted by the controllers who came in through the normal channels. 
Moreover, since the program is not always reaching those individual 
minority members who might be considered to need the most help, che 
methods of recruiting and selection might be examined. There is some 
evidence of a high drop-out rate in the program (Appendix G). 

D. PROFICIENCY TRAINING 

Proficiency training is conducted to reinforce previously learned 
skills or to develop new skills required because of new or revised 
procedures, regulations or equipment." Proficiency training is a con- 
tinuous program which results from the administration of job-centered 
evaluations and examinations. At least one refresher unit (1 hour) 
per month and one facility-developed refresher training package 
(2 hour's) per quarter are to be covered in the minimum 20 -hour annual 
skill maintenance program required of each ATC specialist. This 
training might increase to as much as 60 or more hours annually for 
an individual, depending on how his supervisjr assesses his needs. 
This training covers new procedures, remedial training, and reviews 
of air traffic matters, as required. 



"Much of the following material is taken directly or paraphrased 
from FAJi (1971c). 
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1. Refresher Training 

Refresher training is given to systematically review current 
operational procedures and techniques. It consists of self -study 
units as well as classroom and Ivriefing sessions. The self -study 
units, designed to take about 1 hour each, are developed by the FAA 
Academy generally, or by the facility when training in local oper- 
ational procedures is required. Each specialist is expected to 
complete an average of at least one study unit per month. The units 
are chosen by the facility chief. 

2. Supplemental Training 

Supplemental training is conducted to develop new skills to 
maintain proficiency at assigned operational positions whenever a 
significant change occurs in procedures, regulations, or equipment. 
Each specialist is briefed on national and local changes before they 
become effective. The facilities use a combination of classroom 
training, briefings, and self -study to accomplish supplemental train- 
ing. Training materials are developed centrally or locally, depend- 
ing on the nature of the change. 

3. Remedial Training 

Remedial training is given to correct specific operational defi- 
ciencies, either in response to a disciplinary need (e.g., a system 
error) or in response to appraisals of a controller's performance 
under the Technical Performance Appraisal Program. Under this program 
all controllers are -^^equired to take a performance test and a written 
test. Performance is judged by the supervisor or by someone he desig- 
nates, who watches the controller while he is controlling traffic of 
a specified minimum density. This is the well-known over-the-shoulder 
evaluation. If the contrjller is judged to be weak in some area and 
remedial measures cannot be taken on the job by the first-line super- 
visor, the controller is required to take remedial training. Such 
training may include classroom training, on-the-job training, or both, 
as decided upon by the facility chief or the first-line supervisor. 
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Depending on the nature of the requirement for remedial training, re- 
qualification of the controller through appropriate written examin- 
ations and performance- tests may be required. 

E. ACADEMY TRAINING SUPPORT 

In- addition to training developmentals in residence at the 
Academy, the FAA Academy fulfills a major role in the national train- 
ing program, as follows: 

• Provides advice and assistance to field facilities in plan- 
ning, developing and standardizing air traffic training 
programs and courses, and in developing objectives, curricula 
and schedules for air traffic training. 

• Develops personnel standards for admission to air traffic 
training courses. 

• Develops methods for evaluation of personnel performance and 
progress in training. 

• Reviews training at facilities, and reports on the adequacy 
and effectiveness of the programs. 

• Identifies and recommends desirable training for air traf- 
fic career progression. 

• Develops and updates, as necessary, standardized training 
procedures and materials for facility training programs. 

• Develops and updates, as necessary, a recording method to 
reflect the status of air tr^kfjfic control training through- 
out the agency. 

F. FIELD PROGRAM SUPPORT 

The Air Traffic Training Branch of the Academy supports the 
field training program by providing the following services: 
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Developmental Training 

• Develops instructional programmed learning techniques and 
distributes the program of instruction with the National 
Air Traffic Training Program for each option • 

• Develops and distributes the written examinations on the 
instructional plans • 

• Develops and distributes the National Air Traffic Training 
Program, prescribing the phases of developmental training ^ 
and establishing the criteria for successful completion of 
each phase. 

• Develops and distributes training manuals, including ref- 
erence manuals, directed study manuals, anci programmed 
learning manuals. 

Proficiency Training 

• Develops and distributes self -study materials. 

• Develops and distributes interpretive instructional 
materials and aids in support of new procedural documents 
for use in training situations when appropriate. 

• Develops and distributes instructional materials in special 
training projects in support of immediate requirements of 
the Air Traffic Service. 

• Administers special prototype programs in selected air 
traffic facilities. 

• Develops and distributes the appraisal documents used in ^ 
the Performance Appraisal Program. 

Review 

• Reviews, from reports and records, the training accomplished 
at air traffic facilities, and provides reports on the 
adequacy and effectiveness of the training in meeting re- 
quirements. This work is conducted by three sections in the 
Training Branch: 
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^- Qualificat ion Section . The Qualification Section 
handles parts of the terminal and en route quali- 
fication courses as described previously. It also 
handles the predevelopmental program, special train- 
ing, advanced radar training, facility management 
t. xining, military training, and training for con- 
trollers in private industry. 

2- Automation Section . The Automation Section develops, 
reviews, and teaches courses in new hardware. This 
year (1974), the Section is scheduled to complete the 
courses on the National Airspace System, Stage A 
(NAS-A) and the Automated Radar Terminal System (ARTS) 
III. 

3. Development and Revision Section . The Development 
and Revision Section develops new curricula and new 
training materials, changes manuals regularly, and 
develops tests. 

The numbers of personnel involved in each of these functions in 
1974 are shown in Table F-1. 

TABLE F-1. AIR TRAFFIC PERSONNEL AT THE FAA ACADEMY, 1974 



Number 

Instructors (GS-li to GS-13) 122 

En Route (Phase II) 11 

Terminal (Phases II, IV,- V) 4b 
Special (Initial and Indoctrination) 7 
Predevelopmental 7 

Other (Includes FSS, etc./ 35 

■Supervisors (GS-12 to GS-15) 34 
Clerical ^GS-3 to GS-5) 37 

Total 193 
Educational Technology Branch^ 27 



Resident Students (Daily Average) 




Educational Technology Branch has 27 professional educators and 
12 support staff for entire Academy, .z^ir Traffic students this 
year are about one-third of all resident Academy students. 
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G. TRAINING OF NON-FAA PERSONNEL 

The FAA provides orientation and familiarization training at 
facilities to persons not employed by the agency. Those who have 
successfully completed programs equivalent to FAA Academy develop- 
mental training courses may participate in operational activities 
and training programs related to the performance of duties of develop 
mental positions. Persons who possess a certificate recognized by 
the FAA as comparable to the FAA air traffic control specialist 
(ATCS) certificate may participate under supervision in facility 
training programs and activities at any employee level. 

The FAA Acadejrnj^^traj^ns^ both military and foreign personnel. 
Recently, most of'the military students have been Army personnel, 
since the other two Services have their own air traffic training 
facilities . 

Since 1967, the maximum number of non-FAA students at the 
Academy (the sum of beginning and ending classes) was 424 (fourth 
quarter, 1968). The minimum number was 38, occurring a year later. 
More recently, the number has fluctuated between 50 and about 150. 
-Figure show's ' these data graphically and also shows the percentage 
of all students at the Academy that the non-FAA students have rep- 
resented. Generally, non-FAA students make up about 10 percent or 
less of the entire student body, but there were two quarters (1968- 
69) when non-FAA students made up more than 50 percent of all 
students at the Academy. The number of non-FAA students correlates 
negatively (r = -0.375) with the number of FAA students; if the 
surge of FAA students in the first quarter of 1970 is deleted from 
the data, the correlation coefficient becomes -0.575. Thus, in the 
general case, the attendance of non-FAA students tends to smooth the 
load at the Academy. Since non-FAA students are usually only a very 
minor fraction of the entire student population, however, the mag- 
nitude of smoothing is minor. 
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FIGURE F-1 . Number of Non-FAA Air Traffic Students Attending FAA Academy 
and their Percentage of All Students at Academy (Sum of Beginning 
and Ending Classes by Quarter), FY 1968-1974 
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H. TRAINING LORDS AND COSTS 

Marked fluctuations in the numbers of developmentals hired by 
the FAA each quarter since 1967 have led to wide variations in the' 
training loads at the Academy. Figure F-2, taken from the Corson 
Report (Corson et al., 1970), showed the situation in 1970 which that 
Report said should be remedied.'- As shown in Figure F-3, the training 
loads at the FAA have continued to fluctuate into the present time 
frame. From FY 1971 to FY 1974, the number of students in residence 
at any time has varied from about 300 to almost 1200. 

The percentage of students who failed to graduate from the 
Academy has not exceeded 2 percent since the fourth quarter of FY 
1971 (Fig. F-4). Earlier, during FY 1970, it had reached as much as 
22 percent. Overall air traffic student attrition is considerably 
higher j however (Appendix G). 

The cost of training is considered in detail in Appendix G. 
In formal FAA budget submissions, training is estimated to cost 
$43.7 million for FY 1974 and $55.9 million for FY 1975. To these 
figures one should add an estimate for the pay and allowances of the 
developmentals during training and some provision for the overhead 
costs for training at the facilities. A case is made, in Appendix I, 
that the total annual cost of air traffic training could approach 
$120 million. 



See also Fig. 10, p. 60. 
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FIGURE F-2. Fluctuations in the Number of Developmentals Hired by the FAA, 
FY 1967-1970 (Source: Corson eta!., 1970) 
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I. CAREER PROGRESSION OF AIR TRAFFIC SPECIALISTS 

Figure F-5 diagrams the career of a terminal air traffic special- 
ist. The chart is drawn so as to indicate possible alternatives in 
light lines 5 while the current system is depicted with heavy lines. 
The screening examination, an aptitude test, may have several out- 
comes. In the general case, the individual is put on the roster in 
order of his score on the test. If he has some relevant experience 
(say, as a military controller or pilot) he may be eligible for 
higher placement on the list. A few experienced individuals may 
qualify for higher ratings which permit them to proceed immediately 
to Phase III training with a GS-9 rating. On the other hand, members 
of minority groups who do not make a minimum score may be permitted 
to enter at a lower level through the predevelopment (or "ISO") pro- 
gram. The training program has been described earlier, and it will 
not be redescribed here. Several aspects of the diagram in Fig. F-5 
that are relevant to the current training program should be noted, 
however. The first is the limited role of the Academy in screening. 
The Academy does part of the training, and the facility does the 
remainder. The Academy has virtually no role in selecting or washing 
out trainees, however, although it does test them during and at the 
completion of each course. The Academy provides a rating of the 
student— "Outstanding," "Pass" or "Fail"— but cannot dismiss him. 
Rather, he is permitted to return to his facility, where he may be 
tested and given remedial training if it is considered desirable. 
However, some trainees do leave the FAA voluntarily while they are 
students at the Academy. Also, it must be noted here that once a 
controller starts his career, he is required to continue it to the 
highest journeyman grade at the facility that employs him. Thus, a 
non -radar qualified controller cannot stop at that level if he works 
for a radar facility, even though he could be useful. And a GS-12 
journeyman must continue to study to rise to GS-13 if he works for 
the class of facility that has GS-13 controllers. This is called » 
the "up or out" policy. 
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Following his arrival at full journeyman status, the specialist 
may elect to become a facility instructor. This requires that he 
take an instructor's course at the Academy. In some cases where the 
need is justified and the resources exist, the Office of Training 
will give a Region a waiver to do its own instructor training. In 
any case, once an instructor is qualified by the Academy or by his 
Region, he is not tested further before he undertakes instructor 
duties at a facility. His work, however, is monitored by the Eval- 
uation and Proficiency Development Officer (EPDO) at the facility. 
There is still one further step that is available to the air traffic 
specialists-Academy instructor. Academy instructorships aio bid for 
by the journeyman personnel in the Air Traffic Service, and they 
must be recommended by their facility and regional supervisors. Gen- 
erally this is considered to bo a desirable step because it usually 
means a promotion, since an Academy instructor is considered to be a 
prime candidate for a managerial position after his return to his 
facility. For this reason, all Academy instructor candidates are 
required to take the management training course before they take the 
Academy instructor course. 

Figure F-6 shows the career of an en route controller. The 
general scheme is the same, although it can be seen that the en route 
controller receives a relatively small proportion of his training at 
the Academy — only the part involving non-radar procedures and lab- 
oratory. Since the Academy radar facilities are inadequate, radar 
training occurs at the facilities or, sometimes, where new equipment 
is involved, at NAFEC. After he arrives at the journeyman level, 
the career and options for the en route controller are similar to 
those for the terminal controller. 

We shall mention here one other career aspect of the air traffic 
specialist, the "Second Career." In 1972, legislation was passed 
requiring the FAA to set up a scheme for providing training for con- 
trollers who could not continue their work duo to disability. To 
qualify for this training, a controller must have at least 5 years' 
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service as a journeyman. Then, for medical reasons such as hyper- 
tension or ulcers, or an "operational" reason specified by his super- 
visor, the controller will be considered for Seeond Career training. 
It might be noted that there is available a separate federal program 
for job-related disability for any civil servant, but the Second 
Career program is preferred because the benefits are significantly 
better. The primary benefit is 2 years at full salary plus tuition 
and travel to a school for further educatioa. Counseling is used to 
assist the individual in selecting a second career, but there are no 
particular restrictions on the choice of new occupation. Each case 
is considered individually, and judgment is used to dec ide whether a 
controller qualifies for the program and whether a Second Career pro 
gram ohoi(*e is reasonable and wise. 

Approximately bOO controllers are in the Second Career program 
now. It is expected that the number will increase to about 850 by 
the end of TY 197b. Since the cost in current dollars is something 
like $24,000 per year per man, the total annual cost of this program 
can be expected to reach more than $20 million. This cost is con- 
sidered to be part of the FAA air traffic training budget, but it is 
not included in the comparisons of alternative training programs dis 
cussed in Appendix H. 

vJ. RELEVANCE OF TRAINING CURRICULUM TO OPERATIONS 

The National En Route and Terminal Air Traffic* Training^ Program 
are designed to provide controllers with the knowledges and skills 
they need to provide identical ATC services throughout the nation. 
Standardization of procedures across the national airspace is a fund 
amental requirement of the national air traffic control system. 
Therefore, consistency in the coni.ent and method oi training by the 
Academy and the region is virtually an end in itself. Thus, the 
national programs specify, in great detail, the information and pro- 
cedures that are to be taught during each phase of training and the 
goals to be accomplished before the developmental .can move from one 
phase to Lhe next. Means are provided by periodic review, survey, 
and feedback between the Academy and the regions for modifying Lhe 
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course content and training methods as new needs arise. Whatever 
problems have been encountered in assuring standardization between 
the regions or in keeping the curriculum up-to-date are due more to 
the constraints under which training must operate than to limitations 
in the design of the training programs. Training operates, as every- 
one knows, under such constraints as the size of the annual FAA 
budget, fluctuations in the number of trainees from time to time, 
the priority of operations over training, and the impact of the Civil 
Service Commission, the Whitten Amendment, and the Equal Employment 
Opportunity (EEO) program. The training programs receive frequent 
and vigorous reviews within the FAA (e.g.. Southwest Region, June 
1974; Eastern Region, February 1970; Great Lakes Region, November 
1973; Western Region, May 1973; Corson Committee, 1970; Academy Survey, 
April 1974), and it is not likely that any critical comment could be 
made for the ILrst time. There is substantial evidence, from the 
reports noted above, that training is not standardized betweeri cen- 
ters, that the amounts of training time given to various topics differ 
notably between centers, that some procedures specified by the curri- 
culum are inconsistent with others, that some qualification standards 
are open to differences in interpretation, and that the national pro- 
gram specifies training in procedures that do not agree with current 
operational practices. The resolution of such problems associated 
with training, which have been identified by the FAA itself, would 
seem to require either adherence to program documents or changing 
training requirements that have outlived their usefulness. In either 
c^ase, the matter rests with the FAA. 

Results of the IDA survey, which was addressed to statistics 
about training ac:tLvities, suggest but do not prove that training 
practices at (enters and terminals are not standardized. The evi- 
dence consists of data on the resources available for training at 
20 centers which responded to the questionnaire. Table F-2 shows 
that, on the average, there are 8.2 developmentals per instructor at 
centers; the ratio varies from 2.3 to 20.7. An average 72 percent 
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of the entire staff are fully qual4-f IpfLjourneymen ; this percentage 
varies from 54 to 85 percent. On the average, 18 percent of the 
developmentals do not complete their training; the range is 4 to 36 
percent loss (an extreme value of 77 percent loss has been excluded 
from both the average and the range). The relation between the in- 
structional resources and the fraction of qualified controllers at 
these centers is shown in Fig. F-7. There is a wide variation in 
both parameters, strongly suggestive of a lack of standardization in 
training resources. 
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In addition, the data show wide variations in training resources 
.at terminals also. For 21 major terminals, the data in Table F-3 
show that the average number of developmentals per instructor is 7.3; 
this ratio varies from 0.86 to 20,8, An average 76 percent of the 
entire staff are fully qualified journeymen, but this varies from 
54 to 92 percent. There seems to be no consistent relationship 
between the fraction of the staff needing developmental training at 
these terminal facilities and the training resources. The relation- 
ship is shown graphically in Fig, F-8, Again, such data suggest 
that there is a lack of standardization in training resources and 
hence in the training itself, 
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PERCENTAGE OF OPERATIONAL CONTROLLERS THAT ARE FULLY QUALIFIED 



FIGURE F-8. Relation Between Instructional Resources at Selected Terminals 
and Fraction of Controllers that are Fully Qualified 
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The relevance of the training material to present operations is ^ 
now considered. On the one hand, the flow of controllers and supervi- 
sors between operational and training duties ensures a high degree of 
relevance between the two, as do the formal surveys conducted by the 
Air Traffic Branch (AAC-930) to evaluate the courses developed by the 
Academy.'*' The evidence suggests that efforts are made to correct the 
deficiencies which come to light as a result of such surveys. The 
remedy for some deficiencies, however, might require substantial addi- 
tions to the Academy staff (e.g., to review the complexity of problems 
used in simulated environmental training) or revisionb in the Program 
Control Document (which requires, for example, training on a two-man 
basis at the Academy, while the controllers are evaluated on a one- 
man basis at their field facility). Action on such problems can only 
be referred to the FAA Headquarters. 

Many of the opposing views about training which reflect the 
perceived needs (primarily but not exclusively operational and train- 
ing needs) of various segments of the FAA family are not readily 
resolved by debate and have, in fact, persisted for long periods of 
time. Rather, an objective method is required to provide the in- 
formation that may be used to resolve understandable differences in 
views about the utility of various aspects of training. 

For example, is two-man training at the Academy detrimental to 
the controller's performance at the field facility? The determination 
is made presently by subjective means, i.e., primarily by the super- 
visor's evaluation of the controller's performance, despite the fact 
that NAFEC studies have shown that objective measurement of the con- 
troller's performance is feasible • The significance of the NAFEC 
work is discussed at length in Appendix E* Here, it remains to be 
said that a variety of problems encountered in current training pro- 
cedures can be resolved by objective means associated primarily with 
valid measures of the controller's performance. These measures still 

7< 

In accordance with FAA Order 3000. 18A, Chapter 8, Section 1. 
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require somo development, but their reliabiiity has been doinonstratGd 
beyond question. They are needed to answer such questions about train- 
ing as the following: 

• What is the appropriate amount of time required to achieve 
qualification at each phase of training? What is the appro- 
priate mix of effort in formal training and OJT? 

• In what order should air traffic problems be presented for 
optimum training? The issue is to determine proper incre- 
ments in level of complexity between problems. 

• Is the order of non-radar training followed by radar training 
the correct one? At issue here is the relative difficulty 

of learning the two operations. Learning theory strongly 
suggests that one a.earns more effectively when proceedin^,.-^ 
from simple to complex tasks, rather than in the reverse 
order. 

• What is the value of training on prototype sectors (e.g., 
Tango) rather than on actual ones? This question can be 
resolved by an experimental comparison and not by further 
discussion based on insufficient information. 

A substantial basis for standardizing the training of controllers 
may be found in the reports submitted by the System Development Corp- 
oration (1971, 1972, 1974). These contain comprehensive descriptions 
of what en route and terminal controllers actually do on the job. 
These task analyses have been confirmed by field trials. They provide 
the means for reviewing and modifying the current curriculum of in- 
struction to ensure its relevance to actual operations. It is not 
anticipated that such reviev should produce major changes in the con- 
tent of training. 

Where questions may arise, it should be pointed out that the FAA 
does not have available the means to resolve key issues which may 
affect the content and format of the training program. Means for 
doing so are potentially available (in primitive form) at NAFEC now, 
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although their further and rapid development would be most desirable. 
The training problems identified here require an experimental approach 
for their solution. In such an approach the general question is, "Is 
this way of training more effective than that?" Partisan points of 
view are helpful in identifying problem areas, but not for resolving 
them. Except for the absence of an experimental point of view for 
dealing with training problems, the FAA shows a commendable, although 
primarily subjective, concern for the improvement of its methods of 
training controllers. It is not too much to say that the FAA train- 
ing program can be made more efficient and effective by adopting 
experimental methods, now close at hand, for resolving some of the 
key issues in the present method of training. 

K. SUMMARY 

The various phases of training that in four years transform an 
apprentice into a fully qualified journeyman, as an en route or 
terminal specialist, have been described. Other types of training, 
for predevelopmentals and for proficiency purposes, were also con- 
sidered. The role of the Academy in direct training and in support 
of training at the facilities has been described, and an estimate of 
current training loads and costs has been made. The career progres- 
sions of en route and terminal specialists have been outlined. 

Because of periodic review, the present training curriculum is 
generally relevant to current operations. Nevertheless, some of the 
reviews suggest that training is neither standardized nor completely 
relevant to actual operational practice. This (.ase is also supported 
by data which show marked variations in the number of qualified per- 
sonnel who could serve as instructors and in the ratio of develop- 
mentals to instructors, which also varies among the facilities. Ques- 
tions about the content and/or method of training could be resolved 
by experiments using objective performance measures, a course not now 
being pursued by the FAA. 
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L. CONCLUSIONS 

Training is generally relevant to current operations, although 
some questions exist about the content and method of training. These 
could be resolved by experiments designed to evaluate the significance 
of different ways of training. To be useful, such experiments should 
incorporate objective measures of the controller's performance. Such 
objective measures have been demonstrated at NAFEC (Appendix E). 
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This Appendix presents the costs of training air traffic con- 
trollers in both the en route and terminal options. It also presents 
estimates of the costs of several alternative training programs. 



I. MEASURES OF COST 

Resources devoted to Federal Aviation Administration (FAA) 
training are dominated by manpower, and therefore training costs 
occur almost entirely as salary and benefit expenses. This is also 
true of the alternative training methods considered by this study, 
even for those cases assuming procurement of dedicated simulators. 
Therefore, in this analysis, costs of alternative training systems 
will be considered to" consist only of (1) student travel costs, in- 
cluding per diem, and (2) personnel compensation and benefits (PC&B) 
calculated at Civil Service pay rates in effect at the end of FY 
1974. Facility costs such as rent, utilities, and maintenance are 
not included. 

Criticisms of FAA controller training practices have arisen 
during periods of system expansion due to the seerring incapacity of 
the system to expand its output of qualified controllers. This sug- 
gests two additional related measures of cost for evaluating alter- 
native systems. The first is the elapsed time of controller training 
from accession to qualification. The second is the amount of trainer 
manpower required on the average to qualify new entrants— a measure 
of the labor intensity of the training process. Neither will suf- 
fice independently as a measure of training cost, but each offers 
further means of assessing the net attractiveness of different train- 
ing schemes. 
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The question of manpower required to qualify new controllers 
has not been studied in detail. It is clear that the process is 
heavily manpower intensive, and there may be some ways in which the 
manpower required might be reduced; for example, better selection 
procedures to reduce losses during training, more intensive use of 
simulators using standardized problems and automatic recording of 
performance, and reduction of fluctuations in training load. It is 
believed, however, that there is no good way of evaluating training 
until the FAA develops and places in use an objective method of eval- 
uating controller performance. Two of the alternative schemes of 
training presented in this report (Appendix H) indicate considerable 
savings through a reduction of elapsed time. There are some institu- 
tional traditions against making such changes, but this problem can 
be resolved. The changes need reflect on the quality of neither 
journeyman controllers nor their training. An informal sampling of 
the opinions of informed FAA personnel involved in training elicited 
no dissents fron the idea that the training period could be substan- 
tially shortened. The costing in this Appendix shows the potential 
savings of this and other possible changes in training arrangements. 

II. COSTING METHOD 

Training of controllers occurs both at facilities and at the 
FAA Academy. Since each facility type uses only one controller 
option, there is no problem of cost allocation between en route and 
terminal types of training here. In the case of Academy training 
and training support functions funded through the centralized train- 
ing budget, allocations between the two options must be made. In 
some cases, arbitrary allocations of joint costs have been made. 

Table G-1 lists those activities contributing to costs of 
training of FAA en route and terminal traffic control specialists. 
The last three columns in the table indicate whether the costs are 
whojky identifiable with either terminal or en route controller 
training, or whether the available daca do not permit separation. 
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In the latter case, all costs incurred have been classified as joint 
costs of en route and terminal controller training. 

The items in this list are defined as the explicit costs of 
training for terminal and en route traffic control specialists. Sum- 
med up, they are the total of costs of those resources and activities 
that can be unambiguously associated with training for these special- 
ties. They are also classified as variable or fixed costs, depend- 
ing upon whether they change greatly or little with changes in the 
training workload. Elements classed as variable costs are limited 
to those that will change both significantly and predictably with 
changes in the numbers of persons trained and in the nature of the 
training programs. For variable cost items, functional relationships 
have been formulated for examining the cost implications of alter- 
native training programs. 

The bulk of training costs arises from two elements that are not 
associated with the training function in the FAA budget. These ele- 
ments are the field training staffs (including controller personnel 
on temporary detail to training departments) and controllers engaged 
in training. The total salary bill of all field training staffs is 
roughly twice that of the FAA Academy ^s Air Traffic Branch (ATB). 
Both terminals and en route centers assign controllers to training 
staffs on a temporary or part-time basis. These salary costs are 
approximately equal to those of uhe permanent training staffs. 

The salaries earned by developmental controllers while actually 
engaged in training activities are an unambiguous cost of training, 
regardless of the mode of training — classroom, self -study, or on-the- 
job. When in the classroom or in self -study, the student is engaged 
full-time and is not available for control duties. In on-the-job 
training (OJT), the requirement for continual monitoring by a con- 
troller qualified for the particular position implies double manning 
of the position. One of the two salaries accruing is redundant to 
manning of the position and cannot be considered as a cost of traffic 
control operations. Similarly, the other salary cannot be considered 
a training cost, since it would accrue in the absence of training. 

O 198 

ERIC iM 



The amount of remedial, refresher, and supplementary training of 
full-performance controllers varies widely between facilities. In 
the aggregate, these areas of training constitute 30 percent of total 
trainee man-hours and 5 percent of on-duty time. Facility manpower 
authorization must be large enough to permit manning of control posi- 
tions as well as assignment of qualified controllers for training. 
It is theoretically possible to schedule training during times of ^s? 
light traffic, thereby offsetting at least some of the need for qual- 
ified controllers. However, this can result in a requirement for * 
continuous assignment of instructors to night or weekend work. For 
purposes of this study it is assumed that the increassd manpower 
authorization is necessary for training during peak times. 

An additional source of training cost is the salary bill of 
developmental controllers during periods when they are not engaged 
in training. The present training procedures and scheduler, of train- 
ing extend the period required to attain Journeyman status beyond the 
time that is actually required for training. The result is that 
trainees spend some portion of their qualification period in oper- 
ating control positions for which they are qualified by virtue of 
experience and amount of training completed and in performing other 
facility operating functions. However, the functions they perform 
might be acconplished more cheaply by other means (.even if one accounts 
for the lower salary levels of developmentals) , and there is some 
question whether some of these functions should be performed at all. 
To the extent that facility operating costs are increased by constraints 
on the use of developmentals, training costs are increased, and it 
is clear that same portion of the salaries earned by trainees while 
they are at a facility but are not in training should be considered 
as a training cost. 

The value of trainees to a facility will va^ry with the character- 
istics of the facility (size, type of traffic controlled, etc.), 
the traffic control procedures employed (including control equipment 
used), and the characteristics of the training program itself. For 
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example, in terminal facilities the introduction of National Air- 
space System, Stage A (NAS-A) equipment has significantly decreased 
the requirement for flight data positions previously filled by 
developmentals. What portion should be charged as an implicit cost 
of training is uncertain and can change over time. However, past 
FAA testimony in Congressional hearings supports a contention that 
some part of trainee wages should be considered a cost of training." 
Table G-2 shows costs both with and without this particular implicit 
cost of training. 

< 

A. FIXED COSTS 

1. Instructor Training 

The costs of instructor training consist of personnel compen- 
sation and benefits (PC&B) for- students, teachers, and support staff 
plus student travel and per diem. The data used here were supplied 
by the Educational Technology and Standards (ETS) Branch at the FAA 
Academy as displayed in Table G-3. 

With a teaching staff of 16 and a support staff of two, PC&B 
for the staff are 

1.28C(16)($19,829) + (2)($6836)] = $406,098, 
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'House of Representatives Hearings, 92nd Congress, Second Session, 
Department of Transportation and Related Agencies, Appropriations 
for 1973, Part 3, p. 832: 

Mr. Conte. What do you feel that you have invested in 
^ that person by that time? 

Mr. Flener. We figure 3 years and we probably have $45,000 
wrapped up in him, including his base pay. 

Mr. Flener went on to say that some productivity was obtained from 
trainees, though he gave no specific value. 
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TABLE G-2. COST OF EN ROUTE AND TERMINAL CONTROLLER TRAINING, 
FY 1974 (PERSONNEL COMPENSATION S- BENEFITS, 
TRAVEL, AND PER DIEM) 



En Route Termina l Joint 

A. FIXED COSTS 



1, 


Instructor training^ 


$ 1)26 


$ 410 


$ - 


2. 


Management training (Lawton)^ 


1,995 


1,995 




3. 


FAA Academy Administration and Services 


55 


154 




4. 


Aeronautical Center Support of Academy 


170 


476 


30G 


5. 


Office of Training 


224 


216 




6. 


Regional Training Offices 


512 


495 




7. 


ATB Development/Revision Section 
Non-Radar Support Unit 
Radar Support Unit 




... 


979 




Total Fixed Costs 


$ 3,482 


$ ^,752 


$ 1,384 


VARIABLE COSTS 








1. 


En Route Unit, ATB 


$ 638 


$ - 


$ - 


2. 


Terminal Unit, ATB 




1,663 


— 


3. 


Predevclopmental Training Unit, ATB 






181 


4. 


Field facility training staffs 


, U/ / 


o, Ulo 




5. 


Developmentals at Academy^ 


001 




1 111. 


6. 


Developmentals in training at facilities*^ 


9,284 


17,008 




7. 


Full-performance controllers engaged in 
training and evaluation 


5,63G 


9,965 




8. 


Temporary and part-f me facility training staff 


2,728 


3,317 






Developmental contrujilers; time not in training 


25,179 


19,184 






Total Variable Costs 
(excluding B.9) 


$ 23,134 


$ 37,865 


$ 1,290 




(including B.9) 


$ 48,3r,3 


$ 57,049 


$ 1,29G 




To.tal Training Costs 
(excluding B.9) 


$ 26,666 


$ 41,617 


$ 2,680 




(including B.9) 


$ 51,845 


$ 60,801 


$ 2,680 




Sum of e"i route, terminal, and joint training costs 










(excluding B.9) 




$ 70,963 






(including B.9) 




$115,326 





^This is a summary table. The derivation of all individual entries is explained in correspondingly 
numbered sections of this Appendix. 

^Includes Predevelopmental Training by the FAA Academy. 

^Includes student PC&B, travel, and per diem. 

^Includes travel and per diem. 

^Includes qualification, remedial, refresher, and supplemental training. 
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where 

1.28 represents salary plus a 28 percent benefit rate, 
$19,829 is the GS-12 salary, and 
$6,836 is the GS-3 salary. 

Since there were 8140 student class days, this amounts to $50 per 
student day. 

Student travel and per diom averages $31 per day, calculated as 
follows : 

a . Travel Costs 

Number of facility-priented enrollments x average trip 
cost = 

(286)($200) ^ $57,200 
(NOTE: $200 is the estimate, used throughout this study, 
for average trip cost to the FAA Academy.) 

b. Per Diem Costs 

(Per diem) { ^^Vs per week \ / ^ ^ 

^ ^ ViHstructional days per we'ikj (student class days) 

= ($25)(7/5)(2950) = $103,250 

Thus, total travel and per diem = $57,200 + $103,250 -• $160,450. 
Since there were 5190 student instructor o.ays at the Academy, 
this is an average of $160,450/5190 = $31 per day. 

Finally, PC&B for air traffic students is for grade GS-13: 

1.28($23,433)/230 = $130, 

where 230 is the approximate number of working days per year. 

The total cost of instructor training, then, is 

Staff PC&B $ 50 

Student Travel & Per Diem 31 
Student pc&B 130 

199 ^^^^ student day. 

203 



This amounts to $211 x 446 = $1,095 million for all air traffic 
students during FY 1974. This is allocated to terminal and en route 
specialties on the basis of the FY 1974 proportion of total Air Traf- 
fic Branch (ATB) instructors and facility Evaluation and Proficiency 
Development Officers and Specialists (EDPOs and EPDSs) found in each 
specialty as indicated in Table G-4--48 percent in en route centers 
and 38 percent in terminals : 

$1,095,000 X 0.48 = $526,000 
$1,095,000 X 0.38 = $416,000. 

2. Management Training School (MTS) 

The total cost of MTS in FY 1974 consisted of $1,607 million 
for student travel and per diem and $2,725 million for contract and 
FAA support. This data is based on information provided to IDA in- 
formally and characterized as part of the FY 1974 FAA budget sub- 
mission. 

^ ^The contract and support effort amounts to 

$2,725,000/42,080 = $65 per student day. 

(There were 42,080 MTS student class days, according to Table G-5.) * 

Travel costs are calculated as follows: 

(Number of enrollments )(average cost of travel) ^ 
total daily attendance 

(1 - 0-17) = $20 per student day. 

19 , 459 

The basic formula is multiplied by (1 - 0.17) since about 17 percent 
of the total air traffic enrollments at MTS are air traffic students 
enrolled at the Academy and can be considered to be the same person- 
nel. Therefore, this fraction will not incur any travel cost at MTG. 

The other relevant costs are student PC&B as calculated in item 
1 above, $130 per student day, and per diem, ($25)(7/5) = $35 per 
student day. The total, then, is $65 + $20 + $130 + $35 = $250 per 
student day, or $250 x 19,459 = $4,865 million for all air traffic 
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TABLE G-4. ALLOCATION OF INSTRUCTORS AND EPDOs/EPDSs 



Academy Instructors 
Terminal 
En Route 
FSS 

Predevelopmental 
Special 



Flight Data 

Service Systems 

Station Specialist 

En Route Terminal (FSS) (DSS) Total 



17 



(3.5)= 



47 



(3.5)^ (3.5)" 



(3.5)^ 



47 
17 
9 
7 
7 



Development/Revision 
Radar/Non-Radar 
FSS 



(12)= 



(12 )a 



24 
4 



Automation 



Facility EPDOs/EPDSs 225 



52 



Total 



Percentage 



261 



48% 



159 



205 



38% 



15 
28 

5% 



52 
10% 



52 

379 
546 
100% 



Figures in parentheses are arbitrary allocations between en route 
and terminal options. 

■"individual entries do not total 100% because of rounding. 
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students in management training during FY 1974. This has been allo- 
cated to terminal and en route specialties in the same proportions 
as for total controller personnel authorized to field facilities in 
the FY 1974 budget--41 percent for each (Column D, Table G-6): 

$4,865,000 X 0.41 = $1,995,000. 

3. FAA Academy Administration and Other Services 

The FAA budget submission to Congress for FY 1974 requested 
$1,573 million for support functions of the Academy and operation 
of the General Training Branch. This value has been allocated to 
terminal and en route controller training (and to common support, 
including the predevelopmental training program) as the product of 
two allocation rates. The first rate is the number of staff posi- 
tions requested for the Air Traffic Branch (ATB) as a percentage of 
the staff positions requested for all specialty branches (35%, from 
Table G-7). The second ?ate is derived from the numbers of instruc- 
tional staff posritions in the en route unit of the qualification 
section, the terminal unit, and the predevelopmental program unit 
plus the radar and non-radar units of the development/revision sec- 
tion expressed as percentages of total instructor staffing of the 
ATB (Table G-8). The products are as follows: 

En route: (0.35) (0.10) ($1,573,000) = $55,000 
Terminal: (0.35) (0.28) ($1,573,000) = $154,000 
Common Support: (0.35) (0.18) ($1,573,000) = $99,000. 

4. Aeronautical Center Support of the Academy 

The FAA budget submission to Congress for FY 1974 requested 
$4,858 million for this item. That amount has been allocated to 
terminal and en route training in the same manner as the amount for 
Academy administration (Section I-A-3 above). 

5. Office of Training Support 

The FAA budget submission to Congress for FY 1974 requested 
$829,000 for this item. That amount has been allocated to terminal 
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TABLE G-6 . TOTAL PERSONNEL AUTHORIZATIONS REQUESTED FOR TRAFFIC 
CONTROL FACILITIES AND MAJOR OPERATIONAL PROGRAMS, FY 1974 









i'o:;itions Au- 


Po:; il i'jny An- 


0 




A 




Lhorizod for 
'I'l-al { W CoiM 


l.hoi'i/.cd [or* 
TnHM* CofH i-oi , 


Conl rol \ i*v Po- 
it \k>\v. Author— 




Total I'oKii iony 
AuLhoriy.cd 


at Ccnii'rc f> 
T(*nninah« 


MainlcnarM'i>, f, 
ri iqht Slanthiitls 


i/(Ml loi' I'iold 
Pa< iJ ii i<*:. 




PeryonneJ 




Pci'iJomu'l %^ 


Pi'r.sonncI '/M 


P<'r:;ontu'.l % 


Traffic Control 




'.'^ 








Centers 






ill 




!0,'Jfi'l 41 


Terminals 






jo,n!iO 


Mi,nso ?f» 


]0,H'jO 41 


FJight Scrvico Station:: 










4,r,jn iH 


Central iznl Training 






— 


-- 


— 


Other 


2, Dm 




— 


— 


— 


Maintenance 




2'i 








Field Maintenance 








!0,!>f»!> '?U 


— 


Central i'zctl Training 






— 


— 




Other , 












Flight Standards 




12 








Centralized Training 












Flight Programs C Systems OCM 








4,032 JO 




Other 












Installation & Materiel Services 




' 4 








Centralized Training 


i/ 










Other 












Airports Program 


9/G 


2 








Operations 


mo 










Centralized Ti^aining 


r? 










Other 


2m 










Other 


72C 


J 








Centralized Training 


2 










Other 


72A 










TOTAL 


52,041 


100 


i7,073 100 


41,10li 100 


2fi,Ii0t J 00 



*^In<lividual entries do not add to lOOX because of rounding. 
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TABLE G-7. PERSONNEL AUTHORIZATIONS AT FAA ACADEMY 
BY BRANCH, FY 1974 



Number of Percentage 
Branch Positions Distribution 



Air Traffic 273 35% 

Airway Facilities " , 324 4.1 

Flight Standards 178 23 

Airports 6 1 

TOTAL 781 100% 



TABLE PERSONNEL POSITIONS OF AIR TRAFFIC BRANCH 

BY SECTION AND UNIT, FY 1974 MONTHLY AVERAGES 



Qualification Section 
En Route Unit 
Terminal Unit 

Predevelopmental Training Unit 
Flight Service Station Unit 
Special Unit 

Development /Revision Section 
Radar/Non-Radar Units 
Flight Service Station Unit 

Automation Section 



Number of 

Instructor 

Positions 



17 
47 
7 
9 
7 

24 

4 



87 



28 



52 



167 



Percentage 
of Branch 



10% 
28 

4 

5 

4 



14 
2 



16 



31 
100% 



^Individual entries do not total 100% because of ro-iding. 
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and en route training on the basis of the number of positions re- 
quested for FY 1974 for traffic control in en route centers and 
terminals. As percentages of the authorization for traffic controJ 
and maintenance in all field facilities plus those authorized for 
operations of the flight standards program, these allocation figures 
are 27 percent for en route centers and 25 percent for terminals 
(Column C, Table G-6). 

6. Regional Training Office Support 

The FAA budget submission to Congress for FY 1974 requested 
$1,707 million for training program support by regional headquarters 
and NAFEC. That amount has-been partly allocated to terminal and 
en route training on the basis of the number of positions authorized 
for traffic control at centers and terminal facilities. Table G-5, 
Column B, indicates that the centers have 30 percent of the total 
personnel and the terminals have 29 percent. Thus, for this item 
the costs are $512,000 and $495,000, respectively. 

7. Development/Revision Section of ATB 

Information concerning the characteristics of the radar and non- 
radar support units of the Development/Revision Section was obtained 
by conversations with the current section chief. For both Priority 1 
and Pri9rity 2 tasks (currency of documentation of the national pro- 
grams and currency of proficiency evaluation and remedial training 
materials, respectively), commonality between terminal and en route 
training is almost complete, so that allocat-'on of costs between 
them cannot be justified. Furthermore, the section's workloa-i is 
determined primarily by characteristics of the training program and 
not by numbers of controller trainees at either the Academy or field 
facilities, thus making the cost of this section a fixed cost. 

The instructional staff for FY 1974 numbered 24, supported by 
a staff of 4 technical writers, 10 clericals, and 2 unit chiefs. In 
the section chief's judgment, a staff of this size is adequate for 
Priority 1 tasks under the current national program but permits no 
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effort to be devoted to Priority 2 work. (In FY 1974 less than 1 
man-year was devoted to Priority 2.) His further judgment was .that 
Priority 2 tasks require an average of 10 instructional man-years 
plus support personnel. 

Identifiable costs for this section consist solely of PC&B of 
the staff. These costs have been calculated on the basis of actual 
staffing of the section during FY 1974, as follows: 

GS-13 24 X $23,433 = $562,392 

GS-3 10 X $ 6,622 = 66,220 

GS-7 4 X $ 9,969 = 39,876 

GS-14 3 X $29,095 = 87,285 

Total $755,773 

Total plus 28% benefits $967,389 

B. . VARIABLE COSTS 

Variable costs of instruction at the Academy are based on the 
same procedures for both the en route and terminal options and the 
predevelopmental training program. Costs incurred consist of PC&B 
of the instructional and support staffs, student travel and per diem, 
and the PC&B of students during their period of Academy instruction. 

Tables G-9, G-10, and G-11 display operating levels and cost- 
generating characteristics during FY 1974 for the three relevant 
units of the Qualification Section. They are derived from detailed 
data provided by the section chief. PC&B costs for the staff were 
developed from the monthly staffing (Table G-12) of each unit by GS 
grade level. Student per diem and PC&B are based on total daily 
attendance (the latter at salary rates for the GS level normally asso 
ciated with each phase). Travel costs are allocated at a rate of 
$200 per enrollment, except in Phase V of the terminal option. Here, 
no travel is allowed, on the assumption that all students enroll in 
Phases IV and V sequentially. An implicit assumption associated with 
these costs is that enrollees remain in training for the full term 
of each course, and that no failures are recorded by the Academy. 

211 ' . 
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TABLE G-9. EN ROUTE OPTION, PHASE II NON-RADAR 
FY 1974 Academy Operations 



Class Days Conducted 






608 




Enrollments 






202 




Average Daily Attendance 






32 




Total Daily Attendance 






7,676 




Classes Begun 






16 




Length of Course, days 






38 




Average Class Size 






12.6 




Average En Route Unit Staff 






19.1 




(including Supervisors) 










Student Hours^ per Direct Instructor Hour 




3.6 




Average Daily Attendance per 


Staff Member 




1.7 




Cost per Student Day 






$193 




Course Cost per Student 






$7,338 


4 

4 


Class Operating Characteristics 


(Based on Class Size 


of 12) 




Instruc- 


Students 




Instruc- 


tional - 


^ per 


Instruc- 


Student 


tor 


Hours 


Instructor 


tors 


Hours 


Hours 


Classroom 80 


12 


1 


960 


80 


Class/Laboratory 20 


6 


2 


240 


40 


Laboratory Recap 50 


3 


4 


600 


200 


SET Laboratory 114 


3 


4 


1,368 


456 


"Non-Radar Tango Lab 40 


2 


6 


480 


250 








3,648 


1,016 
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TABLE G-10. TERMINAL OPTION, PHASES II, IV, AND V 



FY 1974 Acadomy Opera f ions 



Phase 



II 



Class Days Conducted 
Enrollments 

Average Daily Attendance 

Total Daily Attendance 

Classes Begun 

Length of Course, days 

Average Class Size 

Average Terminal Unit Staff 
(including Supervisors) 

Student Hours per Direct Instructor Hour 

Average Daily Attendance per Staff Member 

Cost per Student Day 

Course Cost per Student 

Class Operating Characteristics 

Instructional Student:; per 



2 Week 


4 Week 


IV 


V 


'It n 




/88 


40'> 




288 


400 


408 




31^ 


45 


24 


b,330 


4,608 


11,200 


G,J20 


3S 


16 


28 


27 


10 


IG 


28 


V> 


IS, 3 


18,0 


14,2 




r>.o 


/.5 


S,G 


4,3 


$179 


$3G8 


$170 


$] /() 


$1,793 


$2,689 


$^1,166 


$2,S!)4 



AJ.1 
Phase?? 



J 09 
27,2l>8 



1>1,G 



2,1 



Hours 



Instructor Instructors 



Student Instructor 
Hours Hours 



Phase II, 2 Week (Based on Class Size of 

Classroom 50 is 

Laboratory 30 3 

Phase II, 4 V/eek (Based on Class Size of 18) 

Classroojn 92 * la 

Laboratory 36 3 

Phase IV (Based on Class Size of 15) 

Classroom 134 15 

Laboratory 90 3 

Phase V (Based on Class Size of 15) 

Classroom 46 15 

Laboratory 74 3 



1 



1 
6 



/50 
450 



1,200 



1,656 
648 
2,304 



2,010 
1,350 
3,360 

690 
1,110 
1,800 



50 
150 
200 

92 
216 
308 



J 34 
450 
584 



46 
370 
416 



During ^h.^t part of year class v^s taught. Whole-year values are 21 and 18. 
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TABLE G-ll. PEEDEVELOPMENTAL TRAINING PROGRAM 



FY 1974 Academy Operations 

Class Days Conducted 
Enrollments 

Average Daily Attendance 
Total Daily Attendance 13,940 

8 

Classes Begun 
Length of Course, days 
Average Class Size 

Average Staff Size (including Supervisors) 
Student Hours per Direct Instructor Hour 20.5 
Average Daily Attendance per Staff Member 7.1 
Cost per Student Day 

Course Cost per Student ^'t^^-^ 



680 
164 
57 



85 
20.5 

8 



TABLE G-12. QUALIFICATION SECTION STAFFING, FV 1974 
(EXCEPT FLIGHT SERVICE AND SPECIAL PROJECTS UNITS) 



July 

August 

September 

October 

November 

December 

January 

February 

March 

April 

May 

June 

Total Man -Months 
Average Staffing 
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Suoervisors/Instructors 






Predevelopmenrai 


En Route 


Terminal 


Training 


Unit 


Unit 


■ Unit 


1/9 


3/40 


1/7 


1/5 


3/47 


1/7 


1/7 


4/49 


1/7 


1/7 


5/52 


1/7 


1/16 


5/47 


1/7 


2/21 


5/49 


' 1/7 


2/22 


5/48 


1/7 


2/24 


5/48 


1/7 


2/23 


5/49 


1/7 


3/23 


5/46 


1/7 


3/25 


5/47 


.1/7 


3/25 


5/42 


1/7 


22/207 


55/564 


12/84 


1.8/17.3 


4.6/47 


1/7 



The variable costs are divided into nine categories and calculated 
as follows: 

1. Academy En Route Unit (Table G-9) 

The Academy en route unit provides Phase II non-radar training 
to developmental controllers in the en route option. The unit is 
assumed to have 1.8* GS-14 supervisors and 17.3* GS-13 instructors 
assigned to this option. Hence, costs are 

1.28[(1.8)($29,095) + (17.3)($24,811 )] 1.035 = $638,025, 

or $638,025/7676 = $83 per student day, where 7676 is the number of 
student days. 

2. Academy Terminal Unit (Table G-10) 

The Academy provides training in Phases II, IV, and V of the 
terminal option. At staff costs of $61 per student day, the total 
cost of this unit's staff is 



3. Academy Predevfilopmental Training Unit (Table G-11) 

The predevelopmental training unit has 7 GS-11 instructors and 
1 GS-12 supervisor. With a 5% allowance for clerical, th|^s amounts 
to $13 per student day. Since there were 13,940 student days, one 
has the total for this unit as 



4. Field Facility Training Staff 

The costs of the full-time field facility training staff are 
calculated by using data from the IDA facility survey as shown in 
Tables G-13, G-14, and G-15 (pp. 221-226). The basic calculation is 



'$61 X 27,258 = $1,662,738. - 



* ^13 X 13,940 = $181,220. 



(PC&B)(proportion of controllers involved) 
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Option 


Number of 
Controllers 


Total 
Controllers 


PC&B, 
thousands 


Sample 
Size 


Cost, 
thousands 


En Route 


1937 


3233 


$6,465 


0.95 


$4,077 


Terminal IFR 


791 


2110 , 


$3,553 


0.78 


$1,708 


Terminal VFR 


602 


818 


$1,246 


0.70 


1,310 


Total Terminal 








$3,018 



5. Developmentals at Academy 

The cost of developmentals at the Academy is calculated as 

PC&B + Travel + Per Diem. 

For the en route unit, assuming GS-9 Step 2 salary, one has the 
following: 

PC&B $67 
Per Diem 35 

Travel = 200/38 5 

$107 per student day. 

Since there were 7676 student days, the total cost is 

$107 X 7676 = $821,332. 

For the terminal option, one has the following daily costs 
(Table G-10): 



Phase 


Rating 


Daily Cost 


Student Days 


Total Cost 


Phase II (2 weeks) 


GS-7 


$108 


5,330 


$ 575,640 


Phase II (4 weeks) 


GS-7 


$101 


4,608 


465,408 


Phase IV 


iGS-9 


$107 


11,200 


1,198,400 


Phase V 


GS-9 


$107 


6,120 


654,840 



Total $2,894,288 

For the 150 students, the cost of PC&B, travel, and per diem 
amounts to $80 per student day (GS-5). Since there were 13,940 
student days, one has the total cost as 

$13,940 X 80 = $1,115,200. 
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6. Developmentals at Facilities 

The cost of developmentals at the facilities is calculated as« 
the sum of PC&B for (classroom time + self -study time + on-the-job 
training time + other training time) divided by the sample size. 

For the several options, these data are taken from Tables G-13, 
G-14, and G-15, as appropriate. 



Training Mode 




PC&B, thousands 




En Route 


Terminal IFR 


Terminal VFR 


Classroom 


$3,525 


$1,966 


$ 359 


Self -Study 


390 


2,069 


1,079 


OJT 


3,818 


4,145 


1,395 


Other 


1,087 


890 


933 


Total 


$8,820 


$9,070 


$3,766 


Sample Size 


0.95 


0.78 


0.70 


Corrected Total 


$9,284 


$11,628 


$5,380 



En Route 

Terminal IFR 
Terminal VFR 



Total Terminal $17,008 

7. Full -Performance Controllers in Training and Evaluation 

These data are presented in Tables G-13, G-14, and G-15 and are 
adjusted for sample size to give total cost. 

Option C ost, thousands Sample Size Cost, thousands 

$5,355 0.95 $5,636 

. $6,753 0.78 $8,658 . 

$ 915 0.70 1»307 

. m • -1 $9,965 
Total Terminal ^ ' 

8. Temporary and Part-Time Facilit y Training Staff 

The cost of the temporary and part-time facility training staff 
is calculated as ■ . 

(PC&B) (proportion of controllers involved) (sample size ) ' 

213 

217 • 



ERIC 



Controllers 


Total 


PC&B, 


Sample 


Cost, 


up Lion Far LXGxpa Liny 


L.UnLI OJLJLei o 


LilvJUod 1 lUo 




L llvJUod IlUo 








0 95 


728 


Terminal IFR 1319 


2110 


<*» 7 C C 7 

§ J, bb J 


U. /o 


^ O O /I "7 


Terminal VFR 216 


818 


$1,246 


0.70 


470 


Total Terminal 








$3,317 


9, Developmental Controllers 


, Time Not 


in Training 







Data on nontraining time costs are shown here, but since there 
is some question about whether they should be included, either fully 
or partly, in training cosrs,. the total estimated training costs will 
be shown later both with and without this component. It seems that 
some portion of nontraining time costs should be allocated to train- 
ing, but just what portion cannot be specified from the available 
information (see discussion, p. 199). The calculation here simply 
involves correcting the nontraining time costs, as shown in Tables 
G-13, G-14, and G-15, for sample size: 



Option 


Cost, 
thousands 


Sample Size 


Corrected 

Cost, 
thousands 


En Route 


$23,920 


0.95 


' $25,179 


Terminal IFR 


$12,102 


0.78 


$15,515 


Terminal VFR 


$ 2,568 


0.70 


3,669 


Total Terminal 






$19,184 



III. DATA 

In this Section the several sources of data that were used in 
preparing this Appendix are described and discussed. 

A. DISCUSSIONS WITH FAA AND ACADEMY PERSONNEL 

Face-to-face discu<<5.5ions were held with a number of personnel in 
the Air Traffic Branch at the FAA Academy and in the FAA Office of 
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Training. From such discussions data were obtained on the numbers 
of instructors and support personnel in different assignments at the 
Academy. Also obtained were the distributions of Civil Service 
grades, numbers of students by option and phases, numbers of class 
days, attendance, class sizes, course descriptions, and course 
lengths. From budget discussions and from unpublished documents 
relating to budget requests, information was extracted on general 
allocations to the Academy, the Office of Training, and the Aero- 
nautical Center. 

Some of these data represented the actual situation at the time 
they were received. However, variations through the year in personnel 
complement and other factors, would tend to change the numbers some- 
what from those used. An attempt wdS made to use average data for 
the year rather than for a single instant in time, but this was not 
always possible. The error introduced by using snapshot data rather 
than average data is believed to be inconsequential. 

B. IDA SURVEY OF FAA EN ROUTE CENTER AND TERMINAL FACILITIES 

1. Interpretation Problems 

Because the FAA data that were readily available were insuf- 
ficient for the purposes of this study, the IDA study group undertook 
to make a survey' of the FAA traffic control facilities. A survey 
questionnaire (Appendix B) was developed and mailed out to all field 
facilities in September 1974. This was a first effort to collect 
data, of this type. The results should be interpreted with caution 
because the field facilities do not maintain data in the form that 
was requested, and therefore the completion of the form required 
extensive interpretation by the facility personnel charged with fil- 
ling it out. The resources available to the study team have per- 
mitted a limited field test of the form and verification of the data. 

The survey returns of 19 large en route centers were reviewed 
in detail. Follow-up conversations with their training department 
^personnel revealed wide variations both in training programs and in 
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program administration. Such differences led to differing inter- 
pretations of the data requested. The returns from terminals were 
not given a similar review and follow-up, but one would not expect 
to find greater homogeneity among them than among the returns from 
the en route centers. 

Another reason for treating the results cautiously is that they 
represent only a cross section or snapshot of the situation at the 
time the survey was made. The results provide little information on 
structural relationships in the program, and they do not indicate 
how costs will respond to changes in training requirements and pro- 
cedures. Despite its deficiencies, however, the current survey pro- 
vides the only data base available for assessing the impact of train- 
ing system changes. Its errors appear to be nominal and ishould not 
vitiate the essential findings of this study. 

2. Data 

The data derived from the survey are shown in Tables 6-13, G-14, 
and 6-15. They include both costs and training program character- 
istics according to phase; the data were used to derive the 1974 
facility training costs presented in Table G-2 and the cost estimating 
relationships used to estimate the costs of alternative training 
programs . 

3. Completeness of Survey Sample 

A total of 26 questionnaires was sent to en route centers, in- 
cluding the 20 large continental centers. Replies were received from 
19 large centers* and three small centers in time to be included in 
the data sample. Table 6-13 shows the accumulated data for the 22 
responding centers. If one compares this sample with the personnel 
complements of the centers as reported in the Air Traffic Field 



One reply arrived too late to be included in the computations 
in this Appendix. 
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Facility Employment Report of April 30, 1974, one gets results as 
shown on Table G-16, In this table, the survey data have been ad- 
justed upward by 5 percent, since the total number of full-performance 
controllers as represented in the sample is 95 percent of the number 

TABLE G-13. TRAINING HOURS AND COSTS, EN ROUTE 
OmON, FY 1974 (95' PERCENT SAMPLE) 



CoMpltaent Assigned 
Nuabtr 

Avtrigt G.S* Gridt 
S«1<ry KAtt-AnnuAl 
PCtl-Annu<1*(000) ^ 
. fCII-Hourly 

Total H»n-Hours**(000} 



Full 
PtrfOrMinct 



S.221 
12.8 

tS2,7tt 

9, Sit 



Pre- 
OtvelopMCntal 



181 
* 4.4 

340 



Phase II Phase II Phase III Pha* III Total 
Phase I Non>Radar Kadar Non-Kadar Kadar Ocvelopnental 



654 
6.9 
t,S70 
10, ViO 

t.tOf 



371 
8,0 
11,400 
6,<f /J 

7.01 
S07 



57 
9,6 
I3,7S0 
1,004 

107 



398 
10.2 
I4,&b0 

?,4n 
/4fl 



485 
11,9 

18, ^CO 
1 1, AOS 

9JX 



2,346 

A?, 7C4 

'/,7'l 
4,410 



Classrooai Training 
Student Hours (000) 
Instructor Hours (000) 
Student-Instructor Katio 
Student-Staff Katio 
Student »Ctl (000) 
NuHber of Students 
Nuaber of Classes legun 
Avg. Length of Training (Hrs) 
Weighted Avg. Lngth of Trng (Hrs) 
Avg. Class Size 





37.7 


325,6 


163,5 


26,1 


.2 


.4 


553.3 




5,9 


57,4 


47,6 


4,( 


,2 


,2 


115.9 




6.35 


5. 68 


3.43 


5.63 


1 ,00 


1.83 


4.77 




3. It 


2.t6 


1,72 


£.33 










I/I 




I»I4S 


r,::i 




4 






105 


1,010 


422 


256 


20 


12 






21 


119 


51 


27 


5 


7 






238 


318 


347 


89 


8 


24 






359 


322 


387 


102 


10 


33 






&.0 


s.s 


i.3 


9.i 


4.0 


/,7 





Self Study 

Student Hours (000) 
Instructor Hours (000) 
Student Hrs/lnstructor Hrs 
Student PCtI (000) 
Hunber of Students 
Avg. Length of Trng. (Hrs) 
Uelghtid Avg. Lngth of Trng 



(Hrs) 



OJT 

Student Hours (000) 
Instructor Hours (000) 
Student Hrs/lnstructOr Hrs 
Student PC&B (000) 
Nunber of Students 
Avg. Length of Trng, (Hrs) 

Weighted Avg, Lngth of Trng (Hrs) 

Oetalled to Acadeay Ouring Yr. 
Student Hours (000) 
Student PCtB (000) 

Non-Tralning Tlae 
Total Han-Hours (000) 
Less ClassrooR Hrs (000) 

Acadeny Hours (000) 

Self-Study Hours (000) 

OJT Hours (000) 

Other Training Hrs (000) 
Kon-Tralning Hours (000) 
Non-Tralning PCftB (000) 

Leaving Training Progran 
In Passing Status 
In Falling Status 

Training Failures 
CUssrooa 

OJT 

Discrepancy 

Orop-Out Kate (t) 

Prinarlly Ouring 

Cunulatlve Orop-Out Kate (X) 





7.4 


10.5 


41.9 


0 


0 


0 






l,H 


l,A 


a 


0 


a 




.>•/ 


iii 




0 


0 


0 




42 




172 


0 


0 


0 




200 


90 


161 


0 


0 


0 




176 


74 


244 


0 


0 


0 




5.8 


123.5 


8.3 


36.5 


77.7 


175. 




. V 




1.0 






20., 
















... ^ 




y:.n 


UK 






/,«».'.'» 




22 


8b8 


71 


246 


430 


764 




273 


141 


121 


166 


239 


212 




181 


144 


117 


148 


181 


229 




97 




218 


0 








et.o 






0 








AOC 




4tiU 










340 


i,eo$ 


tt7 


\07 


74H 


UIS 










































































i3t 




4t9 




sy.4 






1,1 Of, 




A, 4:.ti 


$:a4 








28 


49 


57 


12 


55 


115 




24 


40 


15 


9 


11 


60 




4 


9 


42 


3 


44 


55 




2 
1 


11 


45 


0 


0 


0 




2 


0 


2 


33 


46 




/ 


-/ 




/ 


II 


9 



59.7 



ft. CO 
AOO 



426.9 
4i. I 

H, 09 



315 

771 



4,410 
553 
HZ 
60 
427 
I '17 

7i,09l 



.1/ 

r.lnaaroom. 



4 14 u n n 

Ctao»roo» Cla»$roon OJT OJT OJT 

5 it 22 3i 4S 
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TABLE G-13. (Continued) 



Reaedltl ?upple»ent<l Refreiher Ceneril Totil 

°^f:j/E:&nc. ^ 202 5.116 4.932 1 .5S3 11.804 

Avg. Length of Training (Hrs) , ^ 22. 1 41 .5 28.7 75.8 

Weighted Avg. Length of Trng. (Hrs) 5.0 «.5 18.9 46.2 32.5 

Student HOMrs (000) 1.0 217.2 93 7J.8 383.1 

Initructor Hrs (000) "''^ "^'^a. 

Student Hrs/Instructor Hrs — , . ']"'" 
Student fClB (000) 

Developaentals , , -e, 

Ku.ber entering , ^ 218 1.370 1.051 346 --- 

Avg. Length of Triinlng Hrs 48.1 61.5 20.7 27.2 

weighted Avg. Length of Trng. (Hrs) 44.5 53.5 21.5 118.5 49.1 

Student Hours (OOO) ^ 9.7 73.3 22.6 41.0 146.6 

Instructor Hours (OOO) J.tf tf.ff /-A -1^./ 

Student Hours/Instructor Hours — — Z***^ 

Student PC» (000) ^ ^r, uU hin .«M /.iM/ 



CUssroopi Other Other Adalnis- 

Total Qualification Qualification Training tralion Other 

Instructor Man-Months , . 

Full TI.e (CPOS/CfOO) ."7 — — 
Part TUe/Te»portry Oetall 1.2'6 

AllSlatlon by Function (X) 'Vol 'jj.J "9.6 ^zhl 'l5.3 7 7 

Min-Hours by Function (000) ^ ^ 'Ji-C U,n M^.^ n.B 

Adjusted Man-Hours by Function (000) 250^6 t'C^o ^ I6l*c 

PCtB by Function (000) "•'^"^ 



'Includes benefits at 28 percent. 

**Man-Yaar: Full Perforaance 1800 hours 
Oevelopnental 1880 hours 



Note: Itali<3$: Derived Values. 
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TABLE G-14. TRAINING HOURS AND COSTS, TERMINAL 
omON; IFR TOWERS, FY 1974 (78 PERCENT SAMPLE) 



Full frt' 
ptrforainct DcvlopBentii 



fhtit I Phut n Phne III fhise IV Ph»se V 



COBpltBtnt Asstfned 
Nuabtr 

Siliry Ritt-Annui 
PCtl*AnnUt1* (000) 
pC»l-Hour1y 
Totil Kin-Hours** (000) 

CUssrooB Tr«lnlnfl 
Studtnt Hours (000) 
Instructor Hours (000) 
Studtnt.-lnstructor Ritlo 
Studtnt-Stiff R«tio 
Studtnt PCftI (000) 
Nusbtr of Students 
NuHbtr of Cliists Btgun 
Avf, Isnjth 5f CUss«s (Hours) 
Mtiant«o Avg. Lngth of Ciisses 

(Hours) 
Avg. CUss SUt 

Stlf Study 
Stuitnt Hov 

Instructor i. t (000) 
Studtnt Hours/Instructor Hour 
Studtnt PCftI (000) 
MuBbtr of Studtnts 
Av|. Lnfth of TrilnJnQ (Hrs) 
Htithttd AV9. Ln5th of Trng 
(Hours) 

OJT 

Studtnt Hours (000) 
Instructor Hours (000) 
Studtnt Hours/Instructor Hour 
Studtnt PCftI 
NuBbtr of Studtnts 
Av9. Lngth of TriJnJng (Hrs) 
Mtlghttd Avg, Lngth of Trng 
(Hours) 

OttiHtd to Acidtsy 

Studtnt Hours (000) 
Studtnt PCftI (000) 

hon-lrtlnlng TUt 

Totti M»n-Hours (000) 
Ltss-CUssroon Hours (000) 
ACidtsy Hourt (COO) 
Stlf Study Hours (000) 
OJT Hours (000) 
Dthtr Trng. Hours (000) 
Non-Trtlnlng Hours (000) 
Non-Trilnlnf PCftI (DOO) 

Ltivlng TrtfnJng Profria 
In Pissing Stitus . 
In FtlHng Stitus 

TrilnJng FiHurts^ 
Cltssroo»/Self Study 
DJT 

DIscrtpincy 

Drop-Out R*tt (X) 
PrUtrlly During 

CuBulttlvt Drop-Out Ratt (S) 



3,553 

19, ItO 

1 1, HI 
$,39& 



43 
5.1 

Hi 

I, ('.*: 
$0.n 



91 
9.0 

12, I/O 
171,1 



208 
8.1 

3,064 

7,0tf 
a9I,0 



444 

8.6 
III, 100 
f»B/7 

7, -14 
HSi,7 



270 
9.4 

n,A$o 

!'07,0 



AO 



446 

10.8 
ti;S70 
n,77A 

H3ll \ & 



4.4 
4.( 

1,97 

89 
- 76 

71 


28.3 
28.1 

1,01 
Z,07 

s/r 
701 
434 
43 


53.4 
33.4 

i,eo 

3, 29 

79*6 
496 

73 


67. J 
44.7 

Lit 
3,09 

toi 
835 
541 
81 


Jt . 7 

46.7 

,7$ 

m 

472 
81 


57.7 
49^5 
/. 17 
S, 40 

353 
84 


iO 
l,S 


42 
l,C 


(•7 
1,0 


HI 
/..*> 


63 


99 
\,C. 


2.4 

.3 

/n 
52 
122 


38.8 
i.O 

r.vo 
€64 
75 


48.3 

tf.2 

682 
97 


99.6 
IZ,B 

741 
808 
218 


56.9 

* 7,3 

455 

205 


22.4 
2. 9 

till 
398 
63 


47 


if 


72 




/.■:& 


i,C 


5.1 
.0 

56 

107 


17,4 
2. 2 

160 
450 
49 


87.7 

u':tl 
1 ,059 
110 


182.5 
23. 4 

K127 
184 


69.1 
9,9 

700 
136 


;152.1 
t9.i 

762 
196 


91 


39 


fi3 


192 


99 


ZOO 


41 

27.) 

tin 




498 
48.2 




327 
7 J. 2 

GO/ 


297 
35.6 

$AC 


90.8 


171, I 


39!, 0 


«3A,7 


h07,C 




4t,9 


90,9 

CHO 


207, i 


442,9 


%e9,.< 


444.9 


27 
27 
D 


71 
69 
2 


90 
84 

6 


125 
82 
43 


62 
57 
5 


85 
53 
32 


2 
0 


D 
1 


4 

5 


10 
34 


3 
6 


3 
31 


-2 


1 


-3 


• 1 


-4 


-2 


i! 

ClissrooB 


to 

CUssrooB 


7 

ClissroOli 


// 


7 

DJT 


iO 
OJT 




10 


IC 


26 


3/ 


39 



Total 
DevelopBcntal 



1.502 

13, 13' 

2,H%S,H 



248.1 
206.9 
1.20 
r,4e 

I, HOC 



W8.4 

1*34.4 



9l3.y 
6i,9 
7,8 
4, tn 



185.0 



.":.*23.^ 
248.1 
185.0 
268.4 
513.9 
110.1 

1,49^.3 

1.1, ltt:i 



2i0 
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TABLE G-14, (Continued) 



KtMdUl 



Supplenentit 



Refresher 



Gtntnl 



Tot«) 



Othtr Training 
FuH Ftrfora«nct 
Nuaktr Cnttrlna 
Avf. Curttlon (Hours) 
Ut1«htt4 Av|. Ln|th of Trng. 
Stu^tn^^Xourt (000) 
Instructor Hours (000) 
Studtnt Hrs/Instructor Hr» 
Stu^tnt P.CII (000) 

Otvt)opatnt«)s 

Nuaktr^CnttrlnQ 

Avf .v,0ur«t1on (Hours) 
Studtnt Hours (000) 
Instructor Mours (000) 
Studtnt Hrs/Instructor Hour 
Studtnt PCII (000) 



(Hrt) 



35« 
U 

ts 

8.3 
f. f 



151 
U 
I.I 

.X 



3.26( 
S3 

2S8.2 



1.524 
21 

21.5 
Z.t 



3.549 

S3 
47 
U7. 

rH, 

I, on A 



1.188 

76 
79. 
10. 



1.702 

62 
81 

137.4 

IH. I 



S26 
21 
7.1 



571.8 



110.1 

7,0 

H90 



CUssroon Other Other Training 

Totjl QujHf1c*t1on Qu^Hf Icjtlon Irjining Adwinlstrjtion Other 

Instructor M«n-Months ^ « 
rull Tint (EfOO/EfOS) 791 
Mrt TUt/Tt«por«ry OttAll 1.319 

Tot*l 2,110 ^ 

Anoc»t1on by Function (X) 100 U.2 22.0 19.8 28.0 8.0 

Training H*n-Kours by Function (nnn) M/.r 70, i fS.c e?..7 aB,c 

Adjusted Min-Hours by Function (000) ifi.S 101,0 lOO.O $Q.t 

rcil by Function (000) J.iW /.w? hOini 



*Inc1udts btneflts «t 21 percent 

**M«n-Yt«r: Full Ptrforaanct 1100 Hours 
OevelopaentAl 1180 Hours 

Kott: Itatiaat Derived Values. 
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TABLE TRAINING HOURS AND COSTS, TERMINAL 

OPTION, VFR TOWERS, FY 1974 (70 PERCENT SAMPLE) 



Full 
r:"':-«ince 



Pre- 
Oeve1opaent«1 



Phase I Phiie M Phase III Phise lY Phase V 



CoNpltsent Assigntd 
Nu«ber 

Avtragt G.S. Grade 
Salary ^.tte-Annual 
PCtl-Annual* (000) 
PCtl-Hourly 
Total Man-Hours** (000) 

ClassrooN Training 
'Student Hours (000) 
Instructor Hours (000) 
StudiTit-inStructor KaliO 
Studtnt-Staff Katio 
Student fClB (000) 
Nuaber of Students 
Nuaber of Classes legun 
Avg. Lngth of Classes (Hrs) 
Avg. Weighted Lngth of 
Classes (Hrs) 
Ava. Class Size 

Self Study 
Student Hours (000) 
Instructor Hours (000) 
Student Hours/ Instructor Hour 
Student fCtI (000) 
Nuaber of Students 
Avg. Lngth of Irng (Hrs) 
Weighted Avg. Lngth of 
Training (Hrs) 

OJT 

Student Hours (000) 
Instructor Hours (000) 
Student Hrs/lnstructor Hour 
Student fCtI 
NuMber of Students 
Avg. Lngth of Trng (Hours) 
Weighted Avg. Lngth of 
Training (Hrs) 

Detailed to Acadeay 
Student Hours (000) 
Stude*:* PCtI (000) 

Non-Training TUe 
Total Mm Hours (000) 
Less-Classroom Hours (000) 
Acadeay Hours (000) 
Self-Study Hours (000) 
OJT Hours (000) 
Other Trng Hours (000) 
Non-Tralning Hours (000) 
Non-Training PCiB (000) 

Leaving Training Program 
In Paising Status 
In Falling Status 

Training Fal lure^ 
Classrooa/Self Study 
OJT 

Olscrep«ncy 

Orop-out Kate {%) 
Prlaarl 1y. Ouring 

Cumulative Orop-out Rate (f) 



984 

10.5 
15.900 
20,024 

1,771 



50 


67 


123 


263 


19 


(.4 


6.9 


7.2 


8,1 


9.? 


7.540 


9.A70 


10,520 


11,890 


13»240 


483 


846 


1,657 


4,003 


322 




n.o/ 


G. 1/ 


/. .ir. 




g4 


126 


231 


494 


36 



10.1 
2.9 

-f/ 
(I 

42 

77 

149 

i.e 



17.3 

S.J 



HI) 

84 

191 



9. 
2, 

H 
60 
1*0 



47 

32.0 

tin 



ttfi 

10 
7 
3 



43 
42 
1 



48 
44 
4 



79 
66 
13 



3 

9.0 
12,980 
50 

6 



9.7 


15.5 


19.9 


.7 


. 3 


7.9 


11.8 


16.6 


.1 


.1 


i.z-t 




1.7.0 




s.oo 


X. i?. 


2.7.4 


2.00 




i. 14 


:■!' 


too 


Hi, 


h 


.{ 


232 


235 


206 


9 


27 


170 


182 


167 


2 


9 


51 


71 


107 


58 


12 


4S 




9ft 




17. 


1.4 


US. 


1.2 




i.O 


38.6 


39.8 


65.6 


3.4 


.1 




.1. 4 




.6 


0 
















IHtt 




I 


332 


336 


326 


25 


27 


110 


117 


206 


176 


4 


lU 


Uh 


7,01 


ISC 


4 


23.1 


49.4 


108.9 


10.9 


.5 


J. / 


C.G 


}4.C 


/.A 


. / 












/ to 




vv/ 


H'A 


27 


317 


430 


418 


44 


87 


123 


296 


197 


20 


vx 


//£ 


2CI 


7.4« 


IW 




122 




2 


0 




11.8 




.4 


0 




/i; 










7,31.2 


494.4 


35.7 


i. r. 




















































48.7 


AO. 3 


190.9 


I A. 8 


2,7. 




f,/H 


1 , .<•'>/ 


//.V 


1/ 



OevelopMcntal 



525 



10.954 
7,361 



987 



56.2 
39.4 

J. 13 
2.^44 



164.8 

X7.,7. 
'/. 4.i 



202.3 

7,7. 

7.4S 



44.2 



9nv.o 

U.7, 
44.2 
164.8 
202.3 
138.3 
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TABLE G-15. (Continued) 



Kc«edl«l SuppU«cnt«l Refresher Ceneri'i 1ol^\ 

Othir TrAlnlnf 

Full Pirfir«4nc« ^.^ , 532 

NusMr Entirina « 949 1,276 532 

Av9. Ourttlon (Hours) , ^ 25 34 « a' 

Htlghttd Av|. Lngth of Trng. (Hrs) 15 30 24 6 

Student Hours (000) ^ -8 ^J-J ".6 34.0 0 

Instructor Hours (000) -3 ^J*^^ 

Student Hrs/Instructor Hrt " " * 

Student Kll (000) tt s/i ^-O'J . 

OtvelopMntAls 218 

Nu.ber Enterlni 37 401 337 Z » 

Av9. Dttrttlon (Hours) , ^ 26 60 »»« 

Htl9httd Av9. Lngth of Trng. (Hrs) iO nc ^0 -^f ^ ^ 

Student Hours (000) 1-9 46-5 13.6 b.4 iJo J 

Instructor Hours (000) ^'"^ J^'* 



Studtnt Hours/Instructor Hours 
Student PCII (000) V "'^ 



Clissrooa 
Quil If Icitlon 



Other Other Tr*lnin9 

qu4l1flc4tlon Tr«lnln9 Adalnlstritton 



Instructor H«n-Honths 

?ull TUt (EPDO/EPDS) 
Pert Tlae/TtRporAry Oet^ll 
Tot*l , , 

Allocation by Function {%) ^ ^ 
Tr«1n1n9 H«n-Hours by Function (000) 
Adjusted H«n-Hours by Function (000) 
PCtI by Function (000) 



(02 

216 
818 

100 

nr 
nr 



14.2 

I?, 4 

24.0 



30.2 
49,4 



30.2 
40,4 



24.6 

40, i 



*Includes benefits et 28 percent 
**Hin-Ye«r: Full PerforMnce UOO hours 
OevelopMntel ' 1810 hours 

Note: nalU»: Oerlved Velues. 
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TABLE G-16. COMPARISON OF FIELD FACILITY EMPLOYMENT REPORT OF 
30 APRIL 1974 WITH RESULTS OF IDA SURVEY OF EN ROUTE CENTERS 



Field Facility IDA Projected Number Projected Number 

Employment Survey for 100% Survey as Percentage of 

Report > 4/30/74 Returns Returns Employment Report 

Full-Performance 5,523 5,221 5,49G 96% 

Controllers 



Developmentals 


2,534 


2,238 


1,^56 1 










96 


Trainees 


161 


225 


237 ) 




Supervisors^ 


1,582 


1,039 


1,094 




EPDOc/EPDSs 


225 


161 


169 


75 


TOTAL 


10,025 


8,884 


9,352 . 


93% 



^Other than facility chiefs and deputy chiefs. 
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in the field facility employment report. This seems to produce a 
large discrepancy between the report and the survey data in the 
numbers .of supervisors, which may be explainable in terms of defin- 
itional ambiguity, since the questionnaire did not ask explicitly 
for supervisory personnel. The discrepancy in EPDOs and EPDSs can- 
not be thus explained, however. 

A total of 394 questionnaires was also sent to facilities per- 
forming tower functions (VFR towers, non-radar IFR towers, TRACONs, 
tower/TRACONs , RAPCONs, combined station/ towers, and other facility 
types). A total of 339 surveys was returned, 317 of which are rep- 
resented in the data contained in Tables G-14 and G-15. (Returns 
fro!ii 22 RAPCONs were not included in the data sample, since traffic 
data were not available.) The total returns were divided into two 
classes, VFR and IFR, on the basis of training requirements. 

A comparison of the number of returns in each class with the 
total number of field installations leads to the conclusion that the 
reduced sample represents 70 percent of all VFR tower controllers and 
78 percent of controllers in all other terminal facilities. These 
percentages were verified by projecting the total number of control- 
lers in each class and comparing the total with that reported in the 
Field Facility Employment Summary of 30 April 1974. The result of this 
comparison was a 2 percent difference in projected controller com- 
plements (Table G-17). 

4. Student -to-Instructor Ratio 

Instructor hours for classroom training are based upon reported 
' student hours and student-to-instructor ratios. The student -to-staff 
ratio is an adjustment of the student-to-instructor ratio to account 
for total instructor man-hours devoted to classroom teaching as re- 
ported by the individual facilities. For the en route centers as 
well as the Academy, the student -to-staff ratio is approximately one- 
half the student-to-instructor ratio. On the other hand, for both ^ 
types of terminal facilities the student-to-instructor ratio is 
approximately one-half the student -to-staff ratio (Table G-18). 
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There is no basic reason for this reversed ratio in the case of the 
terminal facilities. One suspects that the reported percentage 
breakdown of instructor time devoted to different training functions 
is erroneous in the case of the terminals reported. As a result, 
instructor support hours devoted to self -study, OJT, and other train- 
ing are suspect, since they were derived from the percentage break- 
downs . 

These considerations should not affect the total costs of train- 
ing, however. Salaries of students in training are based upon train- 
ing hours and hourly equivalent PC&B at the reported grade levels of 
developmentals , including a .28 percent benefit rate. Instructor 
salary costs are based upon reported total instructor man-hours and 
a similarly calculated hourly PC&B rate. 

5. Training Program Length 

The survey shows marked differences between facilities in train- 
ing program durations for the same phases and modes of training. This 
is shown by the differences in weighted and unweighted averages of 
lengths of training in each table. The unweighted average is the 
average of the training period (of a given phase and mode) reported 
by each facility: 

£ training period reported 
number of facilities reporting 

The weighted average is the sum, acros«« all facilities, of the product 
(number of students entering x length of training) for each facility 
divided by the total number of students entering training in all , 
facilities: 

S (number of students x length of training) ^ 
E (number of students) 

The difference between these two averages will be most pronounced 
when a relatively small number of students undergoes a training 
period quite different from that undergone by the remainder of 
students . 
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5, Mobility 

Personnel in the Office of Training seem to believe that there 
is quite limited mobility among both qualified and developmental con- 
trollers, especially with regard to personnel transferring between 
facilities of the same option. The results of the survey and follow- 
up conversations with field personnel bring this belief into serious 
doubt. Unfortunately, the survey did not explicitly ask for data on 
this point (the origins of personnel new to the facilities or the 
disposition of those leaving the training programs). If interfacility 
mobility is, in fact, extensive, it carries implications for both 
program costs and program designate take advantage of it. It also 
resurrects an issue raised, in the Corson Report (Corson et al«, 1970): 
standardization of training and control procedures and the staffing o^ 
busy and large city facilities are still viewed as problems by a sig- 
nificant proportion of facility personnel contacted, 

1 . Developmental Time Spent in Other Than Training 

The rationale for including some portion of developmentals ^ 
salaries^ while they are not in actual training has been discussed. 
In addition to the problem of assigning some fraction of these sal- 
aries to training, there is the question of the average time develop- 
mentals spend in "other than training" by phase of training. Non- 
training time estimates were derived indirectly. Total developmental 
man-years were determined on the basis of the total year-end com- 
plement of developmentals* Total training time was subtracted to . 
arrive at total nontraining time and was allocated to training phases 
aooording to the year-end complements. Implicit in this method is 
the assumption that developmentals spend a total time in each phase 
proportional to time spent in actual training. While it is recog- 
nized that there will be differing impacts of the Whitten Amendment 
and of queueing to enter training by phase, the available data did 
not permit measurement of actual times spent* 
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IV. ESTIMATING COSTS OF ALTERNATIVE TRAINING PROGRAMS 

In developing estimates of alternative training programs a dif- 
ferent technique from the 1974 cost procedure discussed previously 
was employed. A standard entry level of GS-7 was assumed, and the 
Whitten Amendment was assumed to be the critical constraint. Thus, 
if the full -performance grade level of a certain type of facility is 
GS-12 or GS-13, a 3-year developmental period is assumed and non- 
training time is taken as the difference between 3 years and the 
.actual training time. For facilities with full -performance GS levels 
of GS-10 or less, a developmental period of 2 years was assumed. Non- 
training time and PC&B costs" were then determined on the basis of 
associating a "characteristic" grade level with each training phase 
and assuming promotions occur once a year. 

This estimation process is designed to make cost comparisons 
between alternative training programs whose principal differences lie 
in where training occurs and in the length of the developmental 
period (from initial hiring to full qualification). In this com- 
parison the differences in costs are considered more significant than 
the levels of cost. 

A. FIXED COSTS 

The costs of a number of organizations and activities necessary 
to a training program but whose levels are considered insensitive to 
training loads are not included in the calculations. Those activities 
include all fixed-cost elements of Table G-2. 



B. VARIABLE COSTS 

Estimates of variable costs for each alternative program are 
prepared by defining each program as a sequence of training blocks, 
each identified with location, duration, and other cost generating 
characteristics. Total variable costs of a program alternative^ equal ^ 
the sum of costs of all blocks defining the program. Costs are 
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estimated on a "per student day" basis and are aggregated across 
duration of block and number of students to obtain total costs. 
Costs of each block are estimated in a similar manner and according 
to categories consistent with Table G-2. The numbers of students 
entering each block of training are. set so that 1000 students suc- 
cessfully complete each of the three training programs (en route, 
terminal VFR, and terminal approach control), assuming the dropout 
rates experienced by each option during FY 1974 (Table G-18). Train- 
ing program total costs for the five alternative training program 
concepts are based on the 1000-graduate level. Table G-19 displays 
the relations employed by the model. Table G-20 gives the meanings 
of the input terms. 

SPECIFICATIONS OF ALTERNATIVES 

Tables G-21 and G-22 list the specifications that were assumed for 
the alternative programs described in Appendix H. In addition to the 
items displf cd in those tables, the following are to be noted: 

• Instructor GS grades were assumed as f ollov;s : 

En route centers — GS-13 
IFR terminals--GS-12 
VFR terminals --GS-11 
All Academy--GS-13. 

• A benefits rate of 28 percent was assumed for all students 
and instructors. 

• A per diem rate of $25 was assumed, regardless of the dur- 
ation of continuous training. A travel rate of $200 was 
assumed for each trip to the Academy. For the accelerated 
Academy alternative, a single trip was assumed. For the 
extended Academy and present program alternatives, a trip 
was assumed for each phase with the exception of terminal 
Phase V, where instruction was assumed contiguous with Phase 
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TABLE G-19. OUTPUTS AND RELATIONSHIPS 
Student PC&B per Student Day (Daily Attendanc e) 
^ = ^^-^^^ ( ^260^^ ' centralized training 

R = ^'f^^ > local training 

Student Per Diem per Student Day (Daily Attendance) 

S = ^ ^ E , centralized training, 

S = 0 , local training 

Student Travel Cost per Student Day (Daily Attendance) 
F 

T = , centralized training 
T = 0 , local training 
Instructional & Support Staff Cost per Student Day (Daily Attendance) 



Q = 



(1 - B)(1.28A) 1 
260 I 



Training Cost per Student Day (Daily Attendance) 

U = Q+ R + S + T 
Training Cost per Student (Successfully) Completing Training Block 

Training Cost of All Students (Successfully) Completing Training Block 
/ W = V J 

Total Costs per Student Generated by Training Block, Including Student 
PC&B Incurred During Nontraining Time 



Y = V + 



/ l - H\ p / 1.28C\ 
\^ H / ^ \2080 I 



(1 - K) 



Total Costs Generated by Training Block, Including Student PC&B 
Incurred Daring Nontraining Time 

Z = YJ 

Instructor Man-Year Requirements Generated by All Students in Train- 
ing Block 



^ \1 - 1/ \D/ \2080/ 
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• The terminal VFR program includes Phases I, II, and III 
along with the local control portion of facility check-out 
for the accelerated program alternatives. The terminal IFR 
program covers all phases, including facility check-out for 
both local and approach control. 



TABLE G-20, INPUT TERMS AND THEIR MEANINGS 

Input 

Term ; Meaning 



A Annual salary rate of instructors 

B Support staff cost ratio- -support staff cost as a pro- 

portion of instructional staff PC&B 

C Annual salary rate of students 

D Student -to-staff ratio (student hours/instructor hours) 

E Student per diem rate 

F Travel rate (travel cost per student per training block) 

G Length of training block (hours of training time) 

H Proportion of time in training (ratio of time spent in 

training to total time of training phaseO' 

I Failure/drop-out rate (ratio of students not completing 

training block to those beginning training block) 

J Number of students completing training block 

K Student relative value rate [proportion of student PC&B 

representing students^ worth in traffic control; pro- 
portion (1-K) chargeable to training costs] 
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V. ESTIMATION OF POTENTIAL SAVINGS FROM SEVERAL SOURCES 

A. REDUCTION OF FLUCTUATIONS IN TRAINING LOAD 

Table G-23 displays beginning enrollments at the Academy over a 
period of 28 quarters (7 years). Figure G-1 is a histogram of the 
quarterly data. From the histogram one can conclude there has been 
no typical enrollment level (or central tendency). 

There are no apparent trends or longer term increases or de- 
creases; that is, the fluctuations are short-term (one or two 
quarters) and present a nearly random appearance. The randomness is 
also evident when sequential quarters and years are rank-ordered as 
in the last column of Table G-23. In neither the quarterly nor the 
yearly enrollments are there bunchings of adjacent rank orders. The 
random sequence could well have resulted from drawing the numbers of 
enrollments from a hat. 

In calculating the added costs of capacity to meet such fluc- 
tuating demand, it is assumed that Academy capacity is fixed through- 
out the 7-year period at a level that permits entry of all applicants 
for a given percentage of the time with no delay. Then, several 
percentage levels are chosen* and capacity is calculated as follows 
(Fig: G-2 and Table G-24): 

• 100 percent level . All applicants for entry are admitted; 
the Academy has a capacity to meet the maximum demand experi- 
enced (3100 per quarter). Total enrollment capacity over . 
the 28 quarters would Ije 3100 x 28 = 86,800. Actual enrol- 
ments were close to 23,000, for a 26 percent average use of 
capacity. 

• 90 percent level . All applicants are admitted in 90 percent 
of all quarters. Capacity is IISO per quarter (Fig. G-2). 
Total enrollment capacity over the 28 quarters would be 
32,200. Enrollment demands could not have been met in only 
three of the 28 quarters, and a total of 20,750 would have 
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TABLE G-23. STUDENTS ENTERING FAA ACADEMY BY QUARTER AND 

YEAR, FY 1968-1974^ 

Fiscal Enrollments Enrollment Rank 
year , Quarter Quarterly Annual Quarterly Annual 

1968 1790 . 7 

1 510 22 

2 390 25 

3 490 23 

4 400 24 

1969 2490 r 5 

1 180 28 

2 720 15 

3 390 25 

4 1,200 3 

1970 . 5890 1 

1 3,100 1 

2 660 18 

3 1,400 2 

4 730 14 

1971 3220 4 

1 700 16 

2 670 17 

3 1,020 6- 

4 820 11 

1972 4140 2 

1 - 1,180 4 

2 1,070 5 

3 950 7 

4 940 . ■ 8 

1973 ' 2220 6 

'1 760 13 

2 630 19 

3 300 27 

4 530 21 

1974 3250 3 

1-910 9 

2 630 19 

3 900 10 

4 810 • 12 



S « 23,000 
X « 820 



^Source: FAA Academy training progress reports. 
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QUARTERLY ENROLLMENTS 



3100 3200 



FIGURE G-1. Frequency of Occurrence of Numbers of Beginning Enrollments 
at FM Academy by Quarter, FY 1968-1974. 
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PERCENTAGE OF ALL QUARTERS IN WHICH 
ALL APPLICANTS CAN BE ADMITTED 



FIGURE G-2. Capacity of FAA Academy to Admit Students 

Versus Percentage of All Quarters in Which All 
Applicants Can be Admitted, Based on Historical 
Data for 28 Quarters, FY 1968-1974 (Source: 
Table G-23). 
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been trained over the 28 -quarter period, enrollments in 
those three quarters having been chopped off. The ratio of 
20,750 to 32,200 indicates a 64 percent use of capacity and 
implies a potential for training roughly 11,500 additional 
students at the same Academy cost. 

80 percent level . All applicants are admitted in 80 per- 
cent of all quarters. Capacity is 1000 per quarter (Fig. 
G-2). The Academy would have been sized to train 28,000 
over the 28 quarters. Eliminating excesses of enrollments 
over 1000 in any quarter implies that 20,000 could have been 
trained in the 28-quarter period and that the Academy would 
have been used at 72 percent of capacity. 

60 percent level . All applicants are admitted in 60 percent 
of all quarters. Capacity is 800 per quarter (Fig. G-2). 
Here, the total enrollment capacity throughout the 28-quarter 
period would be nearly equal to the actual enrollments. It 
is evident, however.,, that a planned capacity of 800 would 
involve delays in student admissions that could be avoided 
only by smoothing the flow of hires and demands for ^admis- 
sion. If this were done, a planned capacity of about 800 > 
would be a proper level for .the Academy, based on the experi- 
ence of the past 28 quarters. 

TABLE G-24. POSSIBLE ENROLLMENTS AND QUARTERS OF 

UNDERCAPACITY AT FAA ACADEMY AT FOUR LEVELS OF 
CAPACITY, BASED ON HISTORICAL DATA FOR FY 1968-1974^ 

Capacity Level Number of 



(Percentage of All 
Quarters in which 
Enrollment Demand 
Can Be Met) 



Quarters of 
Undercapacity 
(out of 28 
Quarters ) 



Quarterly 
Enrollment 
Capacity 



Total 
Enrollment 
Capacity 
(for 28 Quarters) 



100% 
60% 



0 
3 



3,100 
1,150 
1,000 



86,800 
32>200 



6 



28,000 



12 



800 



22,400 



^Source: Fig. G-2 and Table G-23. 
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The effective (or average actual) level at which the Academy 
was sized during the 28-quarter period is unknown to this writer. 
The 80 percent and 90 percent levels appear to be reasonable guesses 
for upper and lower bounds T The ratios of the costs of the 80 per- 
cent and 90 percent levels to the "efficient capacity level" (i.e., 
800 admissions per quarter) are as follows: 

• 80 percent level . 1000/800 = 1.25 (20 percent of actual 
costs are attributable to fluctuating enrollments) 

• 90 percent level . 1150/800 = 1.44 (30 percent of actual 
cos.ts-are attributable to fluctuating enrollments). 

These values can be considered as minimums, since their derivation 
did not consider the mixes of enrollment demands or faculty quali- 
fications by traffic control option. It can be expected that rel- 
ative fluctuations by option (and hence the costs of excess capacity) 
would be greater when specialty restrictions are considered. 

The Academy costs which are affected by these considerations are 
the fixed costs plus those variable costs associated v;ith the Air 
Traffic Branch of the Academy (items A.l, A. 3, A. 4, A. 7, B.l, and 
B.2 in Table G-2. ) The total of these items in FY 1374 was $4.45 
million. The savings by reduction in fluctuation, according to the 
calculations above, would have been somewhere between about $900,000 
and $1.4 million for FY 1974. 

B. REDUCTION OF LOSSES DURING TRAINING 

This Section considers the potential .cost savings that might 
accrue from improved selection and assignment. Little information 
is available on the actual effectiveness of various programs that 
might be undertaken to improve selection and assignment. Appendix D 
points out that the inclusion of new tests could increase accuracy 
of assignment from 25 percent (the random possibility of success) to 
58 percent, and that assignments within the IFR terminal and en route 
center options could achieve accuracies of 75 to 80 percent. Even 
such figures are not translatable into training retention rates. 
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One can, however, calculate proportionate potential savings that 
would accrue from eliminating attrition or various proportions of it 
among. trainees • 

Attrition or loss rates include trainees who voluntarily leave 
the service as well as those who are dismissed for cause. The rate 
is the ratio of those leaving to the number entering training, by 
phase. These data are presented in Tables G-13, G-14, and G-15 and 
are summarized in Table It should be noted that the denominator 

of the ratio as usfed heYe "depends on the particular case. In each 
case presented here the denominator is selected from the mode of 
training with the largest number of entrants as identified in the row 
marked "primarily during" in Tables G-13, G-14, and G-15. For en 
route centers, the cumulative dropout rate of the training program 
over the 4-year training period is over 40 percent for new hires at | 
the GS-7 level and nearly 50 percent for those entering through the 
predevelopmental training program. For terminals, the dropout rates 
are also about 40 percent, even for y|]R Jjerminals where training ends 
with Phase III. These rates are as high as or higher than rates 
experienced in the late 1960s. 

The possible cost savings through reduction of trainee losses 
are calculated with the following formula: 

(cost/student)[l - (failure rate)] , 

where the cost per student is calculated according to the method 
developed in Section IV of this Appendix, "Estimating Costs of 
Alternative Training Programs," and where the failure rate is taken 
from Tables G-13, G-14, and G-15, the facility survey data. 

Table G-26 illustrates the calculation for the en route option. 
Each line is calculated according to the formula above. From the 
table it can be seen that cumulative losses increase the cost of 
training an en route controller from $22,235 to $33,163, or by j 
approximately 50 percent. If losses could be completely eliminated, 
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TABLE G-25. CUMULATIVE IMPACT OF DROPOUT RATES 



Phase 


En Route 


I 


II Non-Radar 


II Radar 


III Non-Radar 


III Radar. 


Terminal 


T 


TT 


III 


IV 


V 


En Roul 


:e Centers 












Dropout Rate 


0.05 


0.14 


0.05 


0.13 


0.15 


Rate 


of Success 


0.95 


0.84 


0.95 


0.87 


0.85 


Cum. 


Success Rate 


0.95 


0.80 


0.76 


0.66 


0.56 


Cum. 


Dropout Rate 


0.05 


0.20 


0.24 


0.34 


0.44 


Entries/1000 
Completions . 


1,784 


1,695 


1,423 


1,352 


1,176 


All Terminals 


* 










Dropout Rate 


0.11 


0.09 


0.13 


0.07 


0.09 


Success Rate 


0.89 


. 0.91 


0.87 


0.93 


0.91 


Cum. 


Success Rate 


0.89 


0.81 


0.70 


0.66 


0.60 


Cum. 


Dropout Rate 


0.11 


0.19 


0.30 


0.34 


0.40 


Entries/1000 
Completions 


1,677 


1,493 


1,358 


1,182 


1,099 


IFR Terminals 












Dropout Rate 


0.10 


0.07 


0.11 


0.07 


0.10 


Success Rate 


0.90 


0.93 


0.89 


0.93 


0.90 


-Cum. 


Success Rate 


0.90 


0.84 


0.74 


0.69 


0.62 


Cum. 


Dropout Rate 


0.10 


0.16 


0.26 


0.31 


0.38 


Entries/1000 
Completions 


1,604 


1,443 


1,342 


1,195 
• 


1,111 


VFR Terminals, 












Dropout Rate 


0.13 


0.11 


0.19 






Success Rate 


0.87 


0.89 


0.81 






Cum. 


Success Rate 


0.87 


0.77 


0.63 






Cum. 


Dropout Rate 


0.13 


0.23 


0.37 






Entries/1000 
Completions 


1,594 


1,387 


, 1,235 
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the cost of en route controller 
cent. This is the low point of 
Less than 100 percent reduction 
shown by the rest of the curve, 
reduction of losses of terminal 
"terminal" curve in Fig. G-3. 



training would be reduced by 33 per- 
the "?n route" curve in Fig. G-3. 
of attrition would increase costs as. 
Similar considerations apply to the 
controller trainees, as shown by the 



TABLE G-26. CALCULATION OF COST OF DEVELOPMENTAL 
CONTROLLER LOSS RATES, EN ROUTE OPTION 



Phase 


Place 


Mode 






Calculation 




I 


Facility 


Class 


$ 4,319 


X 


0.95 




= $ 4,103 


I 


Facility 


OJT 


873 


X 


1 




= 873 


II Non-Radar 


Academy 


Class 


8,683 


X 


0.86 


X (410/1423 f 


= 2,152 


II Non -Radar 


Facility 


Class 


7,671 


X 


0.86 


X (1013/1423)^ 


= 4,696 


II Non-Radar 


Facility 


OJT 


210 


X 


1 




= 210 


II Radar 


Facility 


Class 


592 


X 


1 




= 592 


II Radar 


Facility 


Self -Study 


3,643 


X 


0.95 




= 3,461 


III Non-Radar 


Facility 


OJT 


2,607 


X 


0.87 




= 2,268 


III Radar 


Facility 


Self -study 


4,565 


X 


0.85 




= 3,880 






Total 


$33,163 




Total 


per Graduated 


^ $22,235 



student 



Some students in this phase are trained at the Academy, the others at 
facilities. Proportions are used here. 
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AHRITION RATE AS PERCENTAGE OF CURRENT ATTRITION RATE 



FIGURE G-3. 



Potential Reduction In Air Traffic Controller Training 
Costs through Reduction of Attrition During Training. 
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C. REDUCTION OF 'ELAPSED TIME IN TRAINING . 

Training of journeyman air traffic controllers is spread over 
3 to 4 years of time, depending on option and kind of facility. 
During this period the developmental controller spends only a frac- 
tion of his time in actual training,. The remainder is spent in 
assisting the other controllers or in doing other tasks at the 
facility that do not require a journeyman capability. Developmentals 
at facilities are underutilized now, and as automation of facilities 
increases the opportunities for using them productively diminish 
still further. Nevertheless, the developmentals continue to draw 
full salaries for their respective GS levels. Underutilization is 
therefore a cost of training 'that has been discussed earlier. It is 
a cost imposed by the present mode of performing training and season- 
ing, and an indication of the amount of such a cost seems useful. 

In the cost analysis of alternative ways of training, two alter- 
natives were hypothesized to be shorter than the present program, repre- 
senting an elapsed training time equal to the actual training time. 
These data are presented in Table G-27. Subtracting the differences 
between the accelerated programs and the present program shown in 
this table gives possible savings from accelerated training (in 
millions of dollars per 1000 controllers trained) as shown in Table 

The savings are a strong function of the proportion of develop- 
mental nontraining salary assigned to training cost. If no develop- 
mental time is so assigned, the savings are marginal j,nd perhaps even 
negative. However, if all the nontraining time is considered to be 
nonproductive, savings of about $30 million to $48 million per 1000 
controllers trained, depending on option, are indicated ($30,000 to 
$48,000 per trained journeyman). If nontraining time is considered 
to be 50 percent productive, these figures reduce to about $16 mil- 
lion to $26 million per 1000 controllers trained, or a saving of 
$16,000 to $26,000 per trained journeyman. 
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.TABLE G-27. COMPARATIVE COSTS OF ALTERNATIVE WAYS OF 
TRAINING AIR TRAFFIC CONTROLLERS 

(dollars in millions per 1000 controllers trained) 

Percentage of Nont raining Time 



up Lion Or AJLrernQrive 


u^ 




C no/ 


CO/ 

75% 


100% 


En 'Route Centers 












Accelerated Academy 


$33.6 





— 


— 


— 


Extended Academy 


32.3 


$43.4 


$54.5 


$65.6 


$76.7 


Present Program 


29.9 


41.3 


52.6 


64.0 


75.4 


Extended Facility 




41 .1 


52.9 


64.6 


76.3 


HccejLeraLea raciJLxry 


07 0 










IFR Terminals 










IP 


Accelerated Academy 


25.8 


— 


— 


— 


— 


Extended Academy 


29.5 


39.9 


50.2 


60.6 


70.9 


Present Program 


24.2 


34.3 


44.3 


• 54.4 


64.5 


Extended Facility 


22.9 


33.0 


43.1 


53.1 


63.2 


Accelerated Facility 


18.7 










VFR Terminals 












Accelerated Academy 


10.6 










Extended Academy 


15.7 


22.9 


30.2 


37.4 


44.7 


Present Program 


12.8 


19.8 


26.9 


33.9 


41.0 


Extended Facility 


12.5 


19.6 


26.7 


33.7 


40.7 


Accelerated Facility 


8.2 











Formulas for calculating these costs are given in Tables 6-19 and 
G-20. 
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TABLE G-28, SAVINGS OF ACCELERATED CONTROLLER 
TRAINING PROGRAMS 

(dollars in millions per 1000 controllers trained) 



Percentage of Nont raining Time 

Salaries Charged to Training 

Option 0% 25% 50% 75% 100% > 



.En Route Centers 

Accelerated Academy -$3,7 $ 7,7 $19.0 $30,4 $41,8 

Accelerated Facility 2.7 14.1 25.4 3fe.8 48.2 

IFR Terminals 

Accelerated Academy -.I- 6 8.5 18.5 28.6 38.7 

Accelerated Facility 6.3 15.6 25.6 35.7 45.8 

VFR Terminals 

Accelerated Academy 2.2 S.2 16.3 23.3 30.4 

Accelerated Facility 4.3 11.6 18.7 25.7 32.8 



D. USE OF SIMULATORS 

Computers associated with simulators of the types being con- 
sidered for training and refreshing traffic controllers ca.n be pro- 
grammed to provide such bookkeeping functions as grading and main- 
taining records. This would relieve instructors and administrators 
of some chores, but it is not certain that this would amount to much 
in cost savings. Since instructors would be likely to employ any 
time thus gained in additional instruction or in review of simulated 
problems, the major benefits of simulators are much more likely to 
be realized as better opportunities for improving instruction and 
objectively testing candidates, rather than as cost savings. 
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VI. SUMMARY OF FINDINGS 



A, COSTS OF TRAINING CONTROLLERS 

In 1974 the FAA spent something between $61 million and $115 
million on developmental and proficiency training of en route and 
terminal air traffic controllers. These costs almost totally com- 
prise personnel compensation and benefits, travel, and per diem, and 
they do not include costs associated with the procurement or rental 
of facilities. The range of values presented here and in the fol- 
lowing discussion results from whatever assumption is made about the 
allocation of developmental controllers' pay and benefits when they 
are not actually receiving training. The higher cost estimate 
applies if it is assumed that 100 percent of these costs are applied 
to training; the lower cost applies if it is assumed that the develop- 
mental controllers have full productivity and utility commensurate 
with their grade when they are not receiving training. 

The cost of training a single controller is a function of the i 
control option, with the range of costs ag6,in due to the assumptions 
described above. 

Option Minimum Maximum 

En Route $27,000 $77,000 

IFR Terminal 18,000 71,000 

VFR Terminal 8,000 45,000 

Some portion of these costs is incurred through inefficiencies 
in the selection and training system, and a very significant fraction 
is caused by rules under which the FAA is presently operating. 

B. COSTS OF ALTERNATIVE TRAINING METHODS 

Of five alternative training programs for developmental control- 
lers examined in this study,, two offer an opportunity for substantial 
savings. The common characteristic of these two is that they are 
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short, taking only enough elapsed time to cover the formal training 
now being given over a considerably longer period of time. While 
this mode of training could produce substantial savings, there may 
be difficulties in implementing it. Perhaps, in the long run, the 
figures given in Section C below may be useful in helping to bring 
about changes. 

Cost differences between training at the FM Academy and at the 
field facilities are small, with whatever differences there may be 
favoring the facilities because of lower travel and per diem costs. 

C. POTENTIAL SAVINGS FROM VARIOUS SOURCES 

Three possible savings sources are the following: 

1. Smoothing out the flow of students at the FAA Academ y and 
staffing the Academy to deal with a steady load . This could 

- produce a saving of between 20 and 30 percent of the cost 
of Academy operations associated with training developmental 
controllers, which translates into something between $0.3 
million and $1.4 million in 1974. (This range of values 
does not result from the earlivir assumptions about the 
prfjKJuctivity of developmental controllers, but rather from 
assumptions about the staffing level of the Academy,) 

2. Improving selecrion procedures to reduce the losses of 
devexopmentals during training . This could reduce the cost 
of training (per individual trained) by a maximum of 22 per- 
cent for the terminal option and 33 percent for the en route 
option. Since these figures are based on 100 percent re- 
duction of losses during training, any lesser success in 
reducing losses would lower these figures proportionately 
(e.g., a 50 percent reduction of trainee losses would pro- 
vide co55t reductions of about 10 percent in terminal train- 
ing and 16 percent in en route training). 

♦ 
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Acceleration of training and earlier screening (as discussed 
in Section III-C above) . This could reduce training costs per 
individual by as much as $48,000 (1974 costs), depending on 
the option and the assumption made about the degree of pro- 
ductivity of developmentals . when not actually training. 
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This Appendix identifies and evaluates alternative procedures 
currently and potentially available to meet the training needs of 
developmentals • The alternatives reflect differences in degree of 
centralization and differences in the duration of training, both of 
which are issues of concern to the Federal Aviation Administration 
(FAA). 

A- DESCRIPTION OF ALTERNATIVES 

Five alternative approaches to the training of developmentals 
are postulated. The differentiating factors are training location 
and elapsed time of training. The training process and scheduling 
are not considered as significant differentiating variables. In 
each of the alternatives, each student is assumed to receive the 
same training and to undergo the same improved selection process, as 
proposed and discussed in Appendix D. 

1, Accelerated Academy 

In this alternative, the student would receive al his academic 
training at a centralized facility such as the FAA Academy at Okla- 
homa City, The initial training would be continuous and completed 
over a period of approximately 6 to 9 monchs and would include simu- 
lation to prepare the student for final sector or position qualifi- 
cation at his home facility. A capability to simulate both en route 
and terminal facilities would be maintained at the Academy, Separate 
courses would be scheduled for controllers destined for en route, 
instrument flight rules (IFR), and visual flight rules (VFR) facili- 
ties. Screening would be done at the Academy and not take more than 
9 months or so, and it would be done before the trainee achieved 
status as a civil servant. Entry into the Air Traffic Service (ATS) 
would not occur until after^ successful completion of Academy train- 
ing, when status in a Civil Service grade would be assigned. The 
first sector or position qualification would normally take another 
several months at the home facility. Progression to fully qualified 
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journeyman controller would take place as the necessary experience 
and .seasoning are gained* 

2, Extended Academy 

As in the first alternative, all formal training would be given 
at the Academy, but the training for each phase would be followed by 
service and on-the-job training (OJT) at the home (or nearby) facil- 
ity* The overall time would approximate the 3,5- to 4,5-year period 
of the present schedule and the present Civil Service ATC grade 
structure. Screening would be done at the Academy at the completion 
of each training phase. Time to reach final or position qualification 
for journeyman status pr.esxomably would be shorter than in the first 
alternative because of greater familiarity with facility operations. 

3, Present Program 

This is the present prescribed training program, in which the 
Academy and the facilities share the training load. Screening would 
be done primarily at each facility. 

4. Extended Facility 

This alternative poses the possibility of performing all train- 
ing at each facility (or in local groupings, where more appropriate). 
The training plan and pTOgression would be centrally developed and 
standardized, as would the scheduling and performance of regular in- 
spections. After completion of each phase of developmental training, 
facility service and OJT would follow at the facility. As in Alter- 
native 2, the elapsed time to qualification would take 3,5 to 4.5 
years. Final qualification would be similar to present procedures, 
with screening responsibilities resting with the facilities. 

5. Accelerated Facility 

This alternative is similar in pace to Alternative 1, but train- 
ing ib done in a continuous, decentralized fashion at centers and 
terminals (or groups of terminals, as appropriate). This reflects 
the objective of completing formal academic training before proceed- 
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ing to OJT and before Civil Service status is achieved. Course mate- 
rials, plans, and final examinations would be centrally prepared. 
Upon passing, the student would be admitted to the ATS with an appro- 
priate Civil Service rating. The initial period of training would 
take 6 to 9 months, as in Alternative 1. Thereafter, the student 
would commence to gain final sector and position training and experi- 
ence, and he would normally qualify in another few months. 

It is recognized that this alternative would be feasible for 
teiroinal controllers only at the largest terminals. Regional train- 
ing centers could serve the other cases, although centralized train- 
ing would have advantages of. economy and scheduling. 

6. Summary 

These training alternatives are suinmarized in Table H-1. 



TABLE H-1. DEVELOPMEmL TRAINING ALTERNATIVES 
(GS-7 to GS-12/13) 



Alternatives 



Accelerated Extended Present Extended Accelerated 
Academy Academy Program Facility Facility 



Training Duration, 
months 



6-9 



7-10 



7-10 



7-10 



6-9 



Elapsed Time, 



0.5-0.75 



3.5-4.5 3.5-4.5 3.5-4.5 



0.5-0.75 



years 
Final Screening 
Separate Simulator 



Academy Academy Facility Facility 

At Acad- At Acad- No No 
emy emy 



Facility 

Terminal 
simulation 
at Academy 
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B. TRA'INING SPECIFICATIONS FOR EACH ALTERNATIVE 

Underlying the five foregoing alternatives are a number of 
detailed specifications. These include the time and location of 
each phase and subphase of training; the mode of training (i.e., 
classroom/laboratory, CUT, simulation); student -to-staff ratios; 
Civil Service grade classification; and elapsed time in developmental 
status. Tables H-2 and H-3 show the specifications for the training 
of en route and terminal controllers.. As far as possible, procedures 
and times that apply to the present training situation of FY 1974 
are used. For facilities, averages have been derived from the re- 
sults of the IDA survey questionnaire and a consistent set of alter- 
natives has been developed. It is emphasized that these averages 
reflect training in FY 1974, and some differences would be expected . 
in other years. All the details are recorded in Appendix G. 

C. TRAINING COSTS FOR EACH ALTERNATIVE 

An extensive effort has been made to develop cost estimates for 
each of the five alternatives. The workload and costs incurred in 
training apply to FY 1974. The data base underlying all estimates 
derives from FAA budget submissions and data' specifically collected 
for this study from the FAA Academy and the en route and terminal 
facilities. Many of these data have not been collected before. 

The principal cost measure used for cumparison of alternatives 
is the variable training cost per year per thousand graduates. Cap- 
ital costs approximate 10 percent of these variable costs. All the 
details of the costing process are contained in Appendix G, and they 
are an important part of the comparison. 

Table H-4 shows a comparison of the costs of the five alter- 
natives for each controller option. 
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TABLE H-4. VARIABLE TRAINING COSTS PER YEAR PER THOUSAND 
GRADUATES UNDER FIVE TRAINING ALTERNATIVES 5 
EN ROUTE, IFR5 AND VFR OPTIONS 

(dollars in millions) 



Alternative En Route IFR VFR 

Accelerated Academy $33.6 $ 25.8 $ 10.6 

Extended Academy 32.3 29.5 15.7 

Present Program 29.9 24.2 12.8 

Extended Facility 29.4 22.9 12.5 

Accelerated Facility 27.2 18.7 8.2 



The cost differences among alternatives are caused by a complex 
^ set of detailed differences in the specification of each training 
program. However, one major difference between the costs of central- 
ized Academy training and facility training is the per diem and 
travel costs of the former. This cost could be reduced with resi- 
dential facilities operated by the Academy. This and similar con- 
siderations suggest that there may be only minor cost differences 
among the alternatives, except for the question of pay for develop- 
mentals when not training. That question is discussed in Appendix G 
and in Section D, following.. 

D. TRAINEE SALARIES ^ 

The cost estimates in Table H-4 include the full salaries of 
trainees while actually engaged in training. In Alternatives 2, 3, 
and 4 the trainee spends a considerable portion of his developmental 
period performing at his facility those supporting tasks for which 
.he is qualified. Since these are limited, some portion of his 
salary should be considered as a training cost, yet just what frac- 
tion should be assessed against salaries is uncertain. As an example. 
Table H-5 shows the variable training costs for the en route option 
when 0, 25, 50, 75, and 100 .percent of the trainees^ nontraining 
time salaries are -included as training costs. 
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TABLE H-5. " VARIABLE TRAINING COST PER YEAR PER THOUSAND 
GRADUATES, EN ROUTE OPTION UNDER FIVE TRAINING ALTER- 
NATIVES, WHEN VARIOUS. PERCENTAGES OF TRAINEES' NON- 
TRAINING TIME SALARIES ARE CHARGED TO TRAINING 



(dollars in millions) 



Percentage of Nontraining Time 
Salaries Charged to Training 



25 50 7S 



Alternative 



0 



100 



Accelerated Academy 
Extended Academy 
Present Program 
Extended Facility 
Accelerated Facility 



$33.6 
32.3 



$43.4 
41.3 



$54.5 
52.6 



$65.6 
64.0 



$76.7 
75.4 



29.9 



29.4 



41.1 



52.9 



64.6 



76.3 



27.2 



The data in the table give an idea of the cost of stretching out 
the training period. In fact, duration of training is a major cost 
factor. Further, it delays the screening process with associated 
expense. (This is discussed in Appendix G. ) 

In summary, then, considering present procedures and cost struc- 
ture, there is no clear-cut choice of training location on the basis 
of cost. There is, however, a cost incentive to reduce the length 
of time over which training occurs and to reduce the time for 
screening. 
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The sources of training funds and control of those funds are 
spread among a number of offices throughout the Federal Aviation 
Administration (FAA). As a result, it is difficult to identify the 
points at which training is supported by the budget and hence to 
identify accurately the full cost of. training. Of more importance, 
the dispersed origins and control of training funds may make it 
difficult to coordinate a national training program. This Appendix 
discusses budgetary sources of funds for training controllers and 
derives an estimate of total training cost based on the FAA's FY 
1974 budget submission to Congress. Finally, problems embedded in 
present institutional arrangements for funding training activities 
are discussed in the light of stated goals. 

A. METHODOLOGY 

The approach adopted is to assume that the general objective of 
the training program is to provide for such goals as a coherent 
training program capable of dealing with significant fluctuations in 
numbers of persons to be trained, the maintenance of prescribed stan-, 
dards of controller training and proficiency throughout the FAA, 
responsiveness of the training system to new requirements, and effi- 
cient trainifiy in terms of resource requirements. Current arrange- 
ments for funding and administering various portions of the train- 
ing program are examined for this purpose. The analysis is quali- 
tative in nature. 

B. FUNDING OF TRAFFIC CONTROL TRAINING 

Table I-l displays the total FAA budget submission to Congress 
for FY 1974, organized along functional lines. Training is not 
identified directly in these totals. About $51.5 million in these 
accounts can be identified with training, as shown in Table 1-2. In 
addition, there are funds for supporting field facility training per- 
sonnel (EPD0s/EPDSs'--$9 million) and Aeronautical Center support of 
FAA Academy training ($5 million). This would make the total training 
cost appear to be near $66 million. 
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TABLE 1-2. CENTRALIZED TRAINING ITEMS IDENTIFIED IN 
FAA SUBMISSION FOR OPERATING ACCOUNTS, FY 1974^ 

(dollars in thousands) 



Operation of Air Traffic Control System . $590,514 

Air Traffic Service $537,107 

En Route Centers $218,134 

Terminals 210, 967 

Flight Service Stations 80,033 

Direct Support 27,973 

Centralized Training 24,746^ 

Support & Administration 28,661 



Maintenance of Air Traffic Control System 299,528 

Airway Facility Service 262,698 

Centralized Training . 12,935 

Support & Administration (including 23,895 
Development Direction) 

Flight Standards Program 159^920 

Operation of Program 113,436 

Centralized Training 13,255 

Support & Administration 33,229 

Installation & Material Services 116,928 

Material & Procurement 48,186^ 

-Leased Telecommunications & 49,828 

Commercial Services 

Centralized Training 539 

Support & Administratibn 18,375 



$1,166,8.90 



^Source: FAA budget submission to Congress for FY 1974 • 

^Includes $10,900,000 allocation for second-career training • 
c 

Includes supplies, spares, repair, and overhaul for air traffic 
control, navigation, and agency aircraft. 

Predominantly in support of air traffic control. 
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Table 1-3 lists the functions funded from this training budget. 
(The descrepancy of $4 million in training funds between Tables 1-2 
and 1-3 is in part, at least, def itiitional).*^ This information, 
along with other data, provides an estimate of actual training costs. 
There may, of course, also be other methods of estimating the total 
cost of training. Of particular note is that the majority of train- 
ing costs are not shown by budget accounts specifically associated 
with, training. 

TABLE 1-3. ESTIMATED FAA TRAINING BUDGET, FY 1974^ 
(dollars in thousands) 

Office of Training $ 829 

Regional Training Offices 1,475 

FAA Academy 20,128 
Air Traffic Brancli $6,561 
Airways Facilities Branch 6,801 
Flight Standards Branch 4,901 
Airports Branch 127 

other Administration & Support Services 1,738 

Management Training School^ 4,332 

Traffic Safety Institute ^ 109 

Training, Travel 13,834 

Air Traffic Second Career 10,800 

om Aircraft 3yS00 

TOTAL ' ^ $55,116 



^Source: FAA budget submission to Congress for FY 1974. 

^Includes associated contractual, travel, and support requirements. 



The material contained in Table 1-2 comprised part of the formal FAA 
submissions, while Table 1-3 formed a part of informal backup mater- 
ials. Since the two sets of data are structured differently^ the 
sources of discrepancy cannot be pinpointed without additional exam- 
ination. 
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Table 1-4 displays a derivation of aggregate controller training 
costs for FY 1974.* The estimated training budget (of $51 million) 
has been adjusted by deducting items that can be associated with in- 
dependent training activities (those not supported by other headings 
of the general training program) and by adding Aeronautical Center # 
support of the FAA Academy (in particular, supply and facility support). 
From this, the operating budgets of the four specialty training bran- 
ches of the Academy are deducted to define "generalized training 
program support" provided by the estimated training budget. 

The air traffic control training support provided by the training 
budget comprises pro rata shares of generalised training support and 
the Management Training School, along with the total budget of the 
Air Traffic Branch. Training costs not included in the $18.9 million, 
and not identified with training in the FAA budget, include the sal- 
aries of field facility training staffs and the salaries of develop- 
mental and full-performance controllers engaged in training activities. 
Such requirements are clearly a cost of training, regardless of where 
they are funded in the budget. Their magnitude is clearly several 
times that of those training activities estimated as training costs 
above. 

There are both logical and traditional grounds for including as 
a proper cost of training some part of developmental controllers^ 
salaries earned during periods when they are not actively engaged in^ 
training. Logical grounds for this position are discussed in Appen-^ 
dix G. Congressional testimony in 1973 concerning the cost of bring- 
ing new hires up to facility qualification appears to have included 
nearly all of trainees' salaries for the full terms of their 



A detailed construction of controller training costs from Academy 
and field facility operating data is presented in Appendix G. The 
costs presented in Table 1-4 rely, in part, on this material". 



ERIC 



269 

263 



TABLE 1-4. BUDGET SOURCES OF AIR TRAFFIC 
TRAINING RESOURCES, FY 1974 

(dollars in millions) 



Estimated FAA Training Budget * $55.1 

Deductions -18.8 

Om Aircraft $ 3.5 

Aiv? Traffic Second Career 10.9 

Management Training School 4.3 

Trciffic Safety Institute 0.1 

Aeronautical Center Support of FAA Academy 4.9 

Adjusted Training Budget $41.2 

Deductions: Academy Specialty Training Branches Budget -18.4 
Air Traffic $ 6.6 

Airways Facilities 6.8 
Flight Standards 4. '9 * 

Airports 0.1 

CENTRALIZED TRAINING, GENERALIZED TRAINING SUPPORT $22.8 

Air Traffic Branch as Fraction of All $6.6/$18.4 = 0.36 

Specialty Ti»aining Branches 

Air Traffic Branch Portion of Generalized 0.36 x $22.8 $ 8.2 

Training Support 

Air Traffic Branch 

Air Traffic Allocation of Management Training 4.1 
School (includes student PC&B) 

Air Traffic Centralized Budget $18.9 

Other Budget Sources of Air Traffic Training Funds* 59.8 
Field Facility Training Staffs^ $13,1 
Trainee Salaries'^ 46.7 

TOTAL ^ $78.7 



^Includes temporary and part-time training personnel but not OJT 
trainers. See Appendix G. 

^Includes salaries of developmental and full-performance controllers 
while engaged in remedial, supplemental, refresher, and general 
training and salaries of developmentals while engaged in quali- 
fication training. See Appendix G. 
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developmental status.* For FY 1974, the nontraining salaries of 
developmental controllers is indicated to be approximately $44 mil- 
lion. Total controller training cost (nearly all personnel compen- 
sation) for FY 1974 estimated to be between $78 million and $122 
million, depending upon the fraction of developmental salaries in- 
cluded. This is between 5 and 8 percent of the whole budget (not 
grant accounts) of the FAA. It is 7 to 10 percent of FAA operating 
accounts. Further, it is 13 to 21 percent of air traffic control 
personnel costs. Only $27 million can be identified explicitly with 
training functions in formal budget materials and in other FAA 
reports.** 

C. IMPACT OF FUNDING PRACTICES ON TRAINING PROGRAM OPERATION 

There are many FAA activities and offices that control explicit 
and implicit training funds and thereby influence the actual admin- 
istration of the training program. Some of the influences are noted 
below: 

• The FAA Academy is administered by the Aeronautical Center, 
which also controls its budget. The budget allocated to the 
Academy for controller training must compete for funds with 
all of the other diverse functions for which the Center is 
responsible. The amount of funds assigned to controller 



House of Representatives Hearings, 93rd Congress, First Session, 
Department of Transportation and Related Agencies Appropriations 
for 1974 , Part p. 100. Testimony was given confirming that "it 
takes about 3 years to train a controller and it co^ts you about 
$45,000 over the time period." The testimony indicates that the 
$45,000 figure applied to training given in FY 1972 and earlier. 
With Civil Service salary levels and other costs of that time, it 
appears impossible to accumulate unit training costs of this mag- 
nitude without including a large portion of the salaries of the 
trainees. 

Centralized Training Budget ($19 million) and EPDO/EPDS field 
facility training staffs. 
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training is not necessarily responsive to needs for such 
FAA controller training. A specific restraint on training 
arises 'from the difficulty of predicting the required number 
of students to be trained, since the number and schedule of 
new hires is controlled by the Regions and fluctuates widely. 
If this continues as it has in the past, it appears almost 
inevitable that the Academy will, from time to time, be unable 
to meet all FAA controller training requirements. Such 
administrative arrangements invite a lack of standardization 
in the training program and in the performance of its fin- 
ished product. 

The Office of Training is the only organization within the 
FAA concerned with agency-wide training. It has no authority, 
however, over funding of training activities or certification 
of the training programs of field facilities. As a result, 
it has little or no control over the training programs admin- 
istered in field facilities, in terms of enforcement of 
training policy, maintenance of training standards, or util- 
ization of training resources. Responsibility for and control 
over these matters is vested in the regional headquarters. 

Individual facilities and the regional offices under which 
they operate exercise authority for hiring new controllers, 
and they decide when hiring and reporting shall occur. There 
is a seasonality in hiring which produces uneven loads for 
the training pipeline. This seasonality is illustrated in 
Fig. I-l and Table 1-5. Hirings are concentrated in the 
fourth quarter of each fiscal year. The fiscal implications 
of fluctuations in hiring are discussed in Appendix G. 

Authorizations for training staff (EPDOs/EPDSs ) are the 
result of decisions at the facility or Regional level. There 
is great disparity among the Regions in the resources avail- 
able for training. From the IDA survey. Table 1-6 shows wide 
variation among the air route traffic control centers in the 
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IO-f.74.26 



Source: FAA RIS PT 3300-5 



FIGURE 1-1. History of ATC Hirings 
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TABLE 1-6. INSTRUCTIONAL, RESOURCES, HIRES AND ATTPTTTom 
DURING TRAINING AT 20 EN ROUTE CENTE^a^ ^^^^'^ 



En Route Center 

Cleveland 

Chicago 

New York 

Atlanta 

Washington 

Indianapolis 

Fort Worth 

Houston 

Memphis 

Jacksonville 

Miami 

Los Angeles 

Kansas City 

Boston 

Oakland 

Albuquerque 

Minneapolis 

Denver 

Seattle 

Salt Lake City 



Percentage of 
Operational Controllers 
that are 
Fully Qualified 

83% 

54 

56 

60 

69 

72 

70 

75 

67 

64 

72 

60 

77 

81 

70 

77 

69 

85 

81 

83 



Developmentals 
per Instructor 



Hires 



Developmental 

Losses 

Number Percentacye 





DO 


11 


11% 


9 4 


XUv 


24 


9 


12.4 


QO 


36 


14 


9n 7 


111 


12 


5 


8.3 


4 


6 


4 


8.0 


45 


16 


13 


7.7 


31 


13 


12 


4.7 


61 


10 


10 


17.0 


69 


13 


12 


13.9 


102 


23 


15 


5.4 


49 


24 


36 


12,2 


25 


18 


13 


6.9 


21 


9 


10 


3.3 


20 


14 


17 


6.2 


29 


21 


21 


3.0 


76 


17 


27 


8.9 


39 


17 


18 


2.9 


38 


11 


26 


3.6 


0 


7 


18 ' 


2.3 


29 


24 


77 



Source: IDA Survey. 
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ratio of students to qualified instructors (including temp- 
orary training complement). This ratio varies from 20.7 to 
-1 at Atlanta to 2.3 to 1 at Salt Lake City. Even larger 



variations prevail at instrument flight rules terminals. 
LaGuardia has a ratio of 40 to 1, whereas Newark and Houston 
have ratios of less than 1 to 1. 
•• The budget of each Region contains funds allocated for travel 
and per diem for training conducted at the Academy. The 
Regions are free to send men to the Academy^for training or 
to train them at their own facilities, however. In the latter 
instance, the regions are permitted to spend the travel and 
per diem for purposes other than training. The discretionary 
nature of these funds invites inconsistency among the Regions 
in their use of the Academy for training. The diversion of 
these funds has an impact on the overall training program 
that is substantially greater than their magnitude would seem 
to indicate. This increases the problem of predicting train- 
ing loads at the Academy, and perhaps it promotes a lack of 
standardization in training procedures among the Regions. 

From a management point of view, the training program is supposed 
to provide standardized training. Unfortunately, the "conditions 
required to assure this result do not exist at the present time. For 
a variety of reasons, there are considerable fluctuations in the 
rates of hiring, both within and between years. This places an un- 
even load on the facilities required for training, in this case 
primarily the Academy. The Regions follow differing, rather than 
consistent, practices in the extent to which they use the Academy to 
train their personnel. This increases the difficulty of predicting 
and therefore providing the required training capability at the 
Academy. Although there is a National Training Program, responsi- 
bility for its implementation in a standard manner is distributed 
between the Regions, the Academy, and the headquarters organization 
of the FAA. Control over all of the expenses required to train 
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controllers is not found in any single office. It also is distrib- 
uted between the Regions, the Academy, and the FAA headquarters. 
Without urging that the present management structure should or should 
not be changed, it seems clear that the present arrangements could 
provide standardized and efficient training only with the greatest 
difficulty. 

D. 'conclusion 

The sources of controller training funds are diverse and not 
explicitly defined in present FAA accounting procedures. About 5 to 
8 percent of FAA operating expenses can be properly attributed to 
controller training. There is apparently great variation among the 
Regions in the allocation of resources to training. A system of 
regular reporting of all expenditures in behalf of training control- 
lers would offer the opportunity for verification of adherence to 
FAA training policies and the opportunity for a more stable, stan- 
dardized, and perhaps efficient training program. Whether there 
should be central control over these expenditures for training is a 
matter of management prerogative that is beyond the scope of this 
finding. 
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